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and affected by the on-site evaluation process for vocational edu- 
cation programs in Illinois. Data, views, concerns, and improvement- 
oriented recommendations were collected from Team Leaders, Team 
Members,' local administrators, teachers, counselors, students and 
community members, as, well as DAVTE.persohnel . Responses to a series 
of four major research questions were sought:' 

♦ » \» 
• 1. What procedures have the.DAVTE undertaken in order to 

evaluate programs and services aimed' a.t LEP populations? 

2. What are other states doing in evaluating services and 
^ programs aimed at LEP populations? 



3. What are the views and concerns of the- DAVTE staff, 
local administrators' and board members, instructors, 
students and community meifibers with respect to the * 
evaluation of programs and services aimed at LEP 
.populations? v * - ^ 

-4. What is the extent and nature of the impact on local 
education agencies that were evaluated in FY 1980' 
and care s'ferving LEP students? * 

• A variety oT procedures were used to collect the -data utilized 
in the study: i ^ ' 

1. Participant-observation, document analysis and inter- 
v4ews were conducted while the principal, investigator 
participated in three .on-site evaluations of local 
education agencies providing programs and services to^ 
LEP students. ^ ^ 

A questionnaire was developed to obtain input from 
state vocational education directors, regarding the 

.current practices of states'^'n evaluating programs . 

■ and services aimed pit LEP students. 

3. Interviews were conducted with 47 Individuals, to 
discover and verify the views and concerns regarding 
' the present on-site evaluation system. The inter- 
views^ included: DAVTE staff, local adminis'trators-, 
instructors, students and community members, i,e., - 
^ employers parents, Advisory Council members, etc, . 

4- A questionnaire was developed to obtain input from 
selected LEA personnel , regarding the impact on 
programs and- services provided to LEP students that 
were evaluated by the TPS in fiscal year 1980. 
Follow-up phone and personal interviews were con- 
• tiucted to acquire a further in-depth perspective 
of the impact. In addition, document analysis was 
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conducted to consider the composition of the on- 
site evaluation team, recommendations made by the 
team, changes in the One and Five Year Plan, and 
^^also the number of LEP, students being served by 
the LEA. ^ * . " 



The discussion, conclusions and recommendations cTfe^Mncluded 
in the executive sunmary/ J 
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jfeHAPTER 1 ■ 
INTRODUCTION ^ 

Vocational education received its first support frffm the federal 
government with the Smith Hughes Act of 1917, which introduced the 

V 

concepts of "manual arts" and "manual training".. Since this fed^al - 
role started, vocational education has provided skill ti.raining to 
many americans and has prepared individuals for productive preers. 




/Tfiis ThifTaT^fellerar support did not assure' vocational education 
programs as being, readily access.ible to the .handicapped, the disad- 
^vantaged, minorities, women, and those with limited English speaking 
ability. Since then, however, the social and caltural milieu of 
America has changed extensively and vocational education has atten^ted 
to increase its accessibility by including programs and services* 
designed for persons with special needs: 

The first federal legislation to designate persons df limited ^ 
En^ish proficiency (LEPj was the Comprehensive Employment and 
Training Act of 1973 (CETA) which identified these individuals as a 
priority target for occupational training. The next major federal 
legislation was the Education Amendments of 1976, Title II - Vocational-* 
Education Act (Public Law 94-482) which consolidated state admini;s- " 
trative authority and required the s.tates to -develop goals and progr^wTis 
for dealing with LEP populations (U,S, Department pf Labor and the • 
U.S. Office of Education, 1977). In addij;ion, states werff^-rfequired- 
to submit five year-plans for^ vocational education- wtiich addres^d 

* o 

the needs of this special pppuTation (,as well as the handicapped ancj* 
^be disadvantaged populations). According to-yurwitz"(1986), the 
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Education Amendments' of 1976 (P.L. 94-482) provided the"'"most defini-. 

■* 

tive statement "of support for action'in this area" (p. 12).* 

Public Law 94-482 specifically provides federal dollars for ^ 

. . : N- 

institutions >that wish to set up and provide bilingual vocational 

trainingno LEP populations. Although federal funds are available 

foJ. bil ingual vocational education consesnsus has not been reached 

as to the exact meaning and-use of the t^rlrT bi 1 ingual vocational . 

educ^fon. Hurwtz (MSO) in <i45<:tiS5if>g tire- term bii^gtra \ ^\/oc d i i o ri d l 

education states the following: ^ 

It has been'used in reference to a variety of vocational 
eduction programs for those whos^ native language is 
' other than English, The term' is used most precisely jn 
^ reference to vocational education programs where a portion 
bf the vocational instruction occurs in a. language other 
than English, and were Ejaglish is also used or taugh^. 
. Generally these programs i{iclude Vocational English as a 
Second Language (VESL), This English instruction is 
considered a vitdl aspect of vocational programs for 
Limited English speakers, whether or not the program is 
technically bilingual , (Hurwitz, 1980, p, 15) , ' ^ 

The major* goal of federally funded bilingual vocatiorial education 

iprdgrams according to the Office of Education are.to; -1) help'an 

adult who speaks little or no English to acquire useful vocational'. 

skills and '2) learn to speak and understand English welTenough to 

compete successfully in the job market. (U S Department of Education, 

1979). • 

V Vocational education programs aimed a.t serving limited English 

proficiency (LEP) populations are a relatively new educational forum. 

Traditionally, students who were not proficient in Englisli were not 

able to participate fn vocational education programs* Public Law 

94-48^ not only identified persons of limited English prbficiency^ 
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as a .national priority but also required a set-aside" of federal funds 
at the state level specifically for handicapped, disadvantaged and 
limited English proficiency students. States are required to set- 
aside 20 percent oT their Federalallotments for vocational education 
for these specfaj programs and services for the disadvantaged, which ' 
in^ljided persons of limited Enjli.^h proficiency. ^O^this 20 percent 
set-aside, states must U5e for the LEP target population, a minimum- 
^ portion- of funds equal to the ratio of the population between 15 and 
24 years of age'with .1 imited English speaking ability to the general 
population of the same age..^ Even thougli these setT^^ide requirpments 
exists, no current r,Qpprting systems -indicate the extent to which 

* , * . . ^ ^ * y - . , ^ 

such^ populations are served, '^according to jthe Report to* Congress .and 

the. President from 'the. Commissi oner of Education and the Secretary 'of 

Labor entitled The Stat'us 'of Bilingual Vocational Training, Fiscal 

Yedr 1977 . Since*these prograins and services are relatively a ne«^ 

approach to meeting tfie neejis of LEP stcR^nts only a ver;y few s^tes 

have developed evaluation models aYid methodologies to meet the demands 

af these innovative educational'programs.. 

Public law 94-482 identified need of LEP individuals as' 

"most acute" (P,U 94-482, Title II, Part B,.Subpart J, Section 181) 

in. regards to vocational training. In Illinois, vocational education 

programs at the high school, adult education, or community college 

have prov.ided thousands of students with marketable skills. However, 

despite these efforts byN^llinois, Lopez-Va^ladez states the 'follow,ing: 

a significant portion of'the population remains virtually 
r unserved by such .programs . This group consists predomi natfely 
Of persons' o.f limited English-speaking ability (LESA). (Ldpez- 
Valadez, 1979, p. 1)^ , ^ 



n 



This statement is verified by the Illihois State Board of^ Education 

(AnnuaJ Report of the Department of Adult, Vocational and Technical 

Education, Fiscal Year 1979) as^being one of it's four major concerns. 

Th6 Annual Report (1979) s^tates the following: ^ 

In some areas of the State, especially in rural and depressed 
areas, the numb^er and type of vocational education opportunities 
• are significantly limited. As a result, alV students within 
the State do not have equal options for developing skills ^ 
V commensurate with their, interests and abilities. ' (Illinois 
State Board of Education, 1979,, p. 22) 

There has been however, a recent effort in Illinois to provide 
more services^ and programs to LEP populations. The Nlinois State 
Board of Education, Department of Adult, Vocational and Technical 
Education (DAVTE) recently funded a study to review One a'nd Five 
^Itear Plans for those districts that had been approved to claim LEP. 
students for 1980. The'study revea,led that 125 of a possible 750' . 
local education agencies (high schools, area vocational centers, and 
community cplleges) have state-approved One and FiveTear Plans for 
servtirg LEP students in vocational education. These recent efforts- 
to provide services and programs aimed at LEP students need to be 
evaluated in order to assure accountability for federal expenditures, * 
as well as future program improvement^ and growth. The evaluation of 
these services arid programs is being accomplished bj^ the DAVTE 'through 
an evaluation system petprred to as the Three Phase System (TPS) for 
Statewide Evaluation o* vLcational Education Programs. While various 
studies have been conducted specifically on the T.^P.So (Norton & 
Watley, 1972; Smith, 1979; and Cheaney, 19&1), it was not known, 
however, what effects this evaTua^^ion system had upori'^' programs and 
services provided to LEP populations. 



PURPOSE OF STUDY ( ' - • ' 

The purpose of the -study was to provide a better understanding 
of the processes used by the DAVTE in evaluating vocational education^ 
programs and services^aimed at lim'ited English proficient (LEP) 
students in Illinois. This was accomplished by providing a- compre- 
hensive picture of the current evaluation processT. Included in the 
body of data will be the views and concerns not only of the Illijiois 
Department of Adult;. Vocational and Technical Education (DAVTE) but 
also those of the local school and community personnel -who are involved 
in or are- affected by the evaluati^on process. Specifically, the 
study focused on the TPS*s effects upon vocational education programs 
serving (LEP) students in the State of Illinois. 
Background and Rationale 

This first formal requirement for evaluation of vocational edu- 
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cation prog ram s^was-^t forth by the Vocational Education Att of 
1964 and i^s subsequent Amendments of 1968/ The Amendments of 1968 
(P.L. 90-576) require each state to participate in the planning and ^ 
ev€fluation of the vocational education programs within their state. 
Specifically, the Vocational Education Amendments of 1968 require 
each state to 'form its own State Advisory Council for Vocational 
Education (SACVE), The' SACVEs are responsi&fe for "evaluating 
vocational programs, services, and 'actlvf ties assisted under the 
title, and publish and distribute the results" (P,L*. 90-576, Section 
104 and again reiterated in P.L, 94-^2), 

A piece^of legislation that succeeded^ the AmendmenCT^f 1968 
and expanded the formal process of evaluation in vocational education 
was the Education Amendments of .1976 (Title II, P,L. 94-482), 
Specifically, the Education Amendments of 1976 require that: 



eacfi state shall, during the five-year period of the state 
plan, evaluate the effectiveness of each program. within the - 
state being assisted with funds available under this act; 
ar|d Jthe results of these evaluations shall be used to revise 
the state*s programs. (PoLo 94-482, Title II, Section 112) 

Under Section 112 of PoL, 94-482 four aspects of local vocational 

'J 

education programs* are to be evaluat;ed. They include 1) planning and 
operational processes, 2) results of student achievement, 3) results^ 
of student employment success, and 4) other results as measured by 
services to special .populations.' The fourth phase focuses o^h the 
results of services provided to the LEP, disadvantaged, handicapped, . 
minorities and women. ^Even though Sefction 112 place's\an emphasis on 
the evaluation of services provided to special populations, it does 
^not give specific detail on how to accomplish this task. 

• In 1969, in an^attempt to comply With Public Law 90-576 (dealing 
with the evaluation of Vocational programs) the Illinois Division of 
Vocational and Technical Education developed a compi?ehensi ve procedure 
referred to as the Three Phase^System for Statewide Evaluation of 
dLcupational Educational Programs (TPS). In the State of Illinois 
the evaluation activities-are carried out by the Department 'of Adupt,' 
Vocational and Technical Education 1[DAVTE) and are reviewed by the 
SACVEo The DAVTE operated the TPS to evaluate all r^eimbur'sed voca- 
tional education programs (including programs serving LEP students) 
at^ least once very five years. A^eview of the TPS revealed that 
there were specific guidelines and questions which address themselves 
directly to programs and services aimed at LEP students^ however, 
the extent to which these elements of the current TPS reflect useful 
information has not been formality or systematically stOdied, The 



extent to 'which the TPS output information (e.gi final reports) has 
been used for pro^gram improvement .relative to LEP students also has 
not been studied. ' ^ x 

^ ' There is a need for conceptual and technicajl development of 
evaluation procedures which, will assist funders,. administrators, 
instructors, students, and community members to better understand 
the complexities of these n^ programs e^nd services. While federfitl 
legislation and regulations have helped to develop criteria for 

^ defining bilingual vocational training, very little is known about 

the operation of these programs, the instructional practices, and 

the ^fects on traihees. An evaluation system should serve to make 

pers,ons involved in vocational education more aware of LEP issues 

and concerns. Evaluation activity aimed at evaluating services pro- 
« 

vided to LEP populations should also serve to increase the awareness 
and emphasis on LEP students within the local education agency.^ In 
addit|pn to obtaining this increased awareness of and emphasis on LEP 
"^students, program developers, administrators, instructors and other 
interested" persons need informative program data to assist them in 
making appropriate changes and improvements in. programs and services 
provided to thitV LEP populations. 

An evaluation of a, vocational education program serving LEP 
students should se^ve many purposes, and go beyond simple examining 
seV^vices that are currently being -provided tO' students. Such an 
evaluation should also determine whether the services provided are 
the ones that are needed by the specific population being served. 

In adijition, it must assess whether the existing services are compre- 

h 

hensive enough to meet the needs of current students as well as^ 



potential LEP students ^Th-i evaluation results might Indicate that 

certain programs migrit be lui^roveci by, adding services, by deleting 

services for which there is no evident need, or 'hy simply adjusting 

already exisiinj 's^t^Ilc^ i<o ihsure that ttaey better serve the needs 
» • 

of LEP students . 

* t 

A program evalujno. which stresses, the o^ral] worth of a' pro- 
gram has the abiiitj U uoirmiunicate to others the ultimate su ^es^es 
and payoffs tfid^ h^/< i-.cu acjiieved. In addition,- whil e stress;ing 
the overdh ^.'^m i>> .gram, tM^r...Ijr/ services and programs can ^ 

oe laerit^^ -.J : j ,v ^ r . i '.dt^ i i ci . ^rarT.icci> to other pro- 

gra.ajs. • <^ v. j' ^ r t [i'y ^Uidcnts are reUtively 

lin.ue-.^;.> . ^ ' ' . , - (- ( ac. ^..jlated to per- 
mit the t>r . . ^^j a p. >inri£'nt errect on program^ 

success/:: .1 j^t-.-^u . Mf ,r . t ^ ot u.e niinimal con- 

ditions i „ ' ^ . ( * , .t/^i-tis (Troike, 

manuscr 1 ^.t , I:- i . . ir^S ;...../.-.Let> and h.eets these 

purposei li.r 
STATEMEM ( F'RQb' t 

At t-''i5 l^.r.r' * l - . ? .r.t^ e? f., is of the 

Illinois TPS upon p^x. f -.?»d .crv:: -5 prov-rtfed to LEP populations. 
Furthermore, little is'-knuw ^onc^rn^ing the evaluation processes of ^ 
other states in evaluating thfese special programs and services. This 
]Sck of information is delaying th& improvement and further fievelop- 
ment of evaluation methodologies that will eventually provide 
d^'ision-makers with reliable anil valid dat^^ The absence of such 
evaluation procedures ostensibly affects both quality and type of 
programming offer&cHIo LEP. students . 

1 n ■ 
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Undoubtedly, a comprehensive evaluation process\ould provide a 
means by whioh individuals involved in these innovative programs^ 
would achieve greater awareness and understanding of the needs of 
their LEP population. Ultimately, evaluation results would assist 
in deci^ion-maj^ing pertaining to the development, maintenance, and 
improvement of all vocational education programs se/^ving LEP students. 
The principal purpose of this study was to examine and develop a more 
complete understanding of the effects of the JPS' upon programs and 
services provided in vocational'education programs throughout the 
state, . . • t ^ 

RESEARCll QUESTIONS . * ' . 

The research questions will assist in the portrayal of different 
views and concerns related to the evaluation of services and programs 
aimed- at LEP students. Specifically, the questions focus on the 
» evaluation effdrts of the DAVTE, concerning the vocational education, 
programs serving LEP >t;udents in the State of Illinois, The research * 
questibns will intlude: 

!• What procedures has the DAVTE undertaken in order to 

evaluate programs and services aimed at LEP populations? 

2, What are other states doing in. evaluating services and 
programs 'aimed at LEP populations? . 

3, What are the views and concerns of the DAVTE staff, local 
administrators and board members, instructors, students' ajid 

J ) community members with respecpta^the evaluation of programs 
and services aimed at LEP populat^ions? 

4, What is the extent and nature of the impact of TPS on 
selected locfil educatio.n agencies. that were evaluated in 
FY 1980 an^^ are serving LEP students?^ 
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DEFINITION^ OF TERMS 

Limited English Proficiency (LEP) 7- any member of\ national 
^qrigin minority who does not speak and understand the English language 
in an* instructional settling well enough to benefit from vocational 
studies to the same extent as a student whose primary language is ^ 
Eng^^ish (Illinois State Board of Education, Rules and Regulations 



for the Administration of Vocational Programs , 1980), (Note Limited 
Engl is^ Spe 
this study) 



English Speaking Ability (LESA) and LEP were used interchangeably in 

V 

English as a Second Language (ESL) the teaching of English to 



persons whose native language is not English. 

^ Vocational English as a Second Language (VESL) -- the teaching 

I 

of special purpose English to LEP/LESA persons which utilizes the 

r 

vocabulary, situations and lexicon specific to a vocational field or 

job (Bijuigual VocationaKEducation Project 1979) , 1 ^ 

Bilingual Vocational Education programs which are designed to 

ejjable person^ with limited English-speaking ability to acquire the 

necessary job skills by us]ng two* languages «as the medium of instruc- 
* • 

tion. An integral part of these programs is the teaching of voca- 
tional English as a second language (Bi^lingual Vocational Education 
Project^ 1979) . ' ' j 

^ Special' Needs Stud'ihts -- an individual who is having difficulty 
succeedins( in a regular ^r special c^ne^r-orfented education progreim 
duft to being disabled, disadvantaged, different linguistic or culti/fal 



backgratrnd and/or dysfunctional school placement, and who requires: 
(a) inci^vidual by prescribed, unique and more (Vowerful teaching tec 
niques; (b) supplemental or supportive services which vefry in type 
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and extent depending oii individual needs-; and (c) additionVl resources 
from society for t>is/her education and for his/her alceptanc^-'by '■ 
society (Phelps, 1976); " « ' ^ 

ASSUMPTIONS 

'^ The study will attempt to provide a better understanding con- 
cerning the effects of the-Three Phase System 'Upon the program^ and.' 
services provided to LEP populations. The study will assume that: 

1, The datik collected through surveys and interviews will .be 
accurate and valid, 

2, The data collected will assist in the illumination and ' ^ 
identification of the issges and concerns leading to some 

^' ' generalizations. 
LIMITATIONS " , 

The study will ^e conducted under these limitations: ' ^ , • 
1. The primary invisstigator has a personal ar^d professional 

conmitment to the conti nued ^provement and growth of 
- programs and servijies serving LEP populations. 
•2. The generalizations that can be made from this particular 
case s^dy to oth^ states are limited because of vts own 
parficular" contents. The case (or unit o'f study) which was 
examined was defined specifically by the nature and charac- 
te'ristics of the iTlTnoiT^hree Phase System f'or' Evaluating 
Vocational Education. ' ' 
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• • CHAPTER II 

t REVIEW OF LITERATURE 

• OVERVIEW 

In order to establish a' foundation ¥or.linderstanding and simul- 
taneously viewing the issues related tlo^bilingual NTocatiOnal edu- 
cation and its evaluation from an overall ^persisective, *it is 
necessary to review the literature in th^. fol lowing specific areas: 
1. l\\e needs of the limited English proficient student ill 
vocational education. 
. 2. The vocational educjjtion programs and services aimed at 
thejimited English proficient student. 
3. The evaluation of vocational education programs arid 

services aimed at the .limited "Encjlish^ students. ^ ' ^ . 
♦ 4. The study of evaluations: meta-evaluation. 
^ Each of these ares are addressed" in Jhe following sections. 
The needs of 'the Limited English Proficiency (LEP) Student in * 
Vocaticmal Education * ' - . * 

The numBer of limited English speaking persons in the United \ 
States has become_jL major- issue* irt^^;^^4^fyiBg, programs passing^ 
legislation and obtaining -funds. Public Law 42-482 recognized *that^ 

a significant amount of the pbpulatidon, (specifically, the LEP 

^ \ I 

population) was not being served by vocational education programs. 

According to 'Public Law 94-482: ' ; ' . 1 - 

"nnllions of cti^tzens*, both chf?dren 'and adults whose efforts 
to profit from vocational training are severely re^stricted 
by their limited English-speaking ability because they come 
from environments where the dominarftsjanguage is- other tftan ^ 

. ~ . . 12 .\ ' ^ ' 
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English; (the fact) that such persons are therefore unable * 
. to help to fill the critical need for nxire and better 
^^>ra'fned personnel in vital occupational categories; and ' 
that such persons are unable to make their maximum con- 
tribution to the Nation's economy and must, in fact, 
suffer the hardships . of unemployment or underemployment 
(.P.Lo 94-482, Title II, Part B, Subpart 3, Sectian 181j ^ 
\ 

There are at least 28 milliqn persons in the United States, including 
about 5 millian school-age children whose dominant lanrguagi is other 
than English. The great majority were born' in the United ^States 
and are^citizens. Mcrre specifically, about two-thirds af all these . 
persons and more than four-fifths of tbe school -age children are, 
'^native born. This data is drawn from the" Survey of Income and 
Education (SIE), conducted by the U,S„ Bureau of the -Census, 

Spring, 19^/ Other findings provided by the SIE indicated that 

/ 

one persons in eight in the United States had a non-English language 
bacjcgpotTnd. Furthermore, the speci/fic language background of more 
than a third of all language minority perso.ns and 60 percent' of the 
, school^age children in this "group was Spanish. Spanisl^-language 
background persons jiumbered 10.6 million. The SIE data also 
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indicated that language minority persons were located *in every 

> 

' State in the Union. However, three out ofVive wer^ocated in 
five States of the Southwest^ Arizona; 'California, Colorado, Nev* 
Mexico, and Texas. These States plus New York, Florida, Illinois,' 
and New Jersey accounted for about 90 percent of the Spanish- 
language background population. The aj)Ove''SIE data provides for 
the first time' in the UoS* history, individual state estimates of 
current language usage as well as language backgrounds. It is 
Indeed important that .this. typje of data be collected since. 
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statistios and statistical analysis have become. a driving force in 

social and educational pollcy-rrfaking. 

In examinij:rg the characteristics of the LEP population in 

Illinois, one finds that the needs are similar to tKat of the 

« 

national population. Illinois i.s a' state with a large population of 
LEP persons. A study conducteJ^by^Lopez-Valadez and Balasubramonian 
(1978) indicatec^ that Illinois had an estimated^ 466, 72l" non-English 
/nl5ther tongue speakers ages 14-24. This number is 19.4 percent of the 
total number of persons^ in Illinois ages 14-24\ Jt was projectejl that 
this percentage would increase to 21.6 percent by January, 1980. 

- - * » 

The following data was als'o drawn from this study: ; ' 



1, Spanish 65,74%, Greek 4,52%, Italian 4.44% and Korean 
.2,28% are the^ largest language groups in Illinois, 

2, LESA population' estimates in various counties of- 
Illinois indicate that Cook County has the largest 
concentratioji _of the 14-24 year old LESA population; 
Champaign, Cook, DuPage, Kane, Lake, Rock Island, 
St, Clair, Wfj^'teside, Will and Winnebago Counties 
'each have a thousand or more LESA persons,^ 

3, Only 13 percent of tHe 14-24 year old non-English 
mother tongue population is enrolled in a college 
or university,^ and OQly 14,8 percent of high school 

students' plan to attend a college or university. 
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Thus, about 72 percent of the 14-^4 year old non- 
English mother tongue population may be in need of 
vocational edttcation regardless of how much of 
thetr mother tongue they use, (Lopez-Valadez 8r 
Balasubramonian, 1978, p. 6) 

These data' vividly demonstrate the need for special vo^:ational 

programs and services to assist the LESA population in Illinois, 

-According tF WorkeTs of Spanish Origin (1978)^ Illinois in 1978 

had a general unemployment rate ot, 5, 4** percent but 8,9 percent for' 

Hispanics, Unemployment is most severe among Hispanic youth ages 

,14 • ( 



♦16-24. They represented nfearly 45 percent of aXl^'' unemployed -Hi spa nfcs 

although they compremise only 26 percent of the total Hispanic labor ' 

force (Workers of Spanish Origin, 1978). 

T|?e improvement of efnployment. opportunities for persons whose ^ 

native-language is' other than English is a major goaf of bijingual 

vocational education/ L'opez-Valadez (1980)* states that"^: 

Despite the fact that traditicu}ailyT^1lfspani been 
tracked ^*nto vocational 'and^hon-coll^ge Bound programs," 
there is ITs^urprising underrepresentation of Hispanics 
in vocational education progr^jTis-, in particular at the<^^ 
post-jecondary level.. ^. This\ow 1-evel of par.ticf pati^on 
by, Hispanics can be attributed to -th^' lah^; of fourMiajoV 
elements:" acQess, counseling, pro gramma t>f cooj::dfnation^ 
and fihaTicial aid. rt6pez-V?il5fdez, 1980, p. 2)/ , 

* * . — . 

_The^faiTuriB of vocational education to meet the. needs of LEP 

.populations necessitates a full commitmei\t on the part of vocational 
educators -to better serve this- population. One of the most viable a 
'alternatives available to vocational, educators is the bilingual ^ 
vocational education program as proposed by P^Lo 42-482. Unfortu- 
nately much of what has been done in this area has gone unnotice 
die to a lack. of national research and dissemination efforts. The 
limited research, development and dissemination efforts according 
to Phelps^^(4980) , have caused a restricted rate of developmenLand 
expansion of vocational ^education programs serving special popul^ations 
The Development of Vocational Education Programs apd Services Aimed 
at LEP Populations j - ^ . 

Education Amendments pf 1974, Part B (Public Law 93-380), identi- 
fled persons with limited English speaking -fbility as a target group 
■for which States were authorized to provide vocational training in 
coordination with the bilingual education programs (Title VH of the 
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El^entary and Secondary Education Ac.t). This authorization for 
States to provide vocational training was a commendabl-e approach, 
h.owever, the States were allowed to set their own priorities for 
the use of fu-nds with the requirement that they Serve the purpose 
ou$,lined by -Congress. * - 

*The Education Amendments of 19^6 (Public Law 94-482) 'provided' 
several changes in rege^rds to programs serving LEP persons\ Public 
Law 94-482 requires the States to consolidate their administrative 
autiiprity. over a vat:iety of types of programs and also required the 
States to submit annual and 5 year plans for vocational education 
which include ^als and programs for dealing with LEP^ersons as 
well as the handicapped and disadvantaged. The allotted funds for 
each^'State is described in the following manner by^-the U»S. Depart- 
ment of Labor and U.So Office of Education: * 

♦The 1976 Amendpients specify three groups fj^r whom a portion 
of the ^illotted funds to each State must be spend: \ disad- 
vantaged persons, persons of limited English-speaking 
ability, and adults, ,The provisions for "National Priority 
• /Programs", Section 110 (b) (1), "specify^hat a State must 
set -aside each fiscal year at least 20 percent pf the^tate's 
allotment for vocational education (Section 102(a)) to pay 
at least half the cost of vocational education for disad- 

vVantaged persons and persons with limited English-speaking 
ability- Each State must use a minimum portion of the 20 
percent set-aside for- vocational education for persons with 
limited English-speaking ability. The minimum portion is 
equal to the ratio of the limited English-speaking population 
the State, aged 15-24 (Section 110 (b) (2)). Thus, if, 10 
percent of the 15- to 24-year-old population in a State has 
limited English-speaking ability, $2 out of every $100 of 
the State's allotment for vocational education must be used 
for persons with limited English-speakfng ability.. (U.S/ 
, Department of Labor and UoSo Office of Education, 1977. p. 12) 

Co r\gress through legislation (Public Law 94-482) appropriated $2,8 

million for bil>figbal vocational training, programs in fiscal years', 

1977, 19Zl^nd 1979-. Fiscal 'yeara978 there were 12"bTTingual t 
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vocational training programs, 3 bilingual vocational instructor pro- 
grams, and one materials development project. 

For FY 1976, the U,S, Department of Labor and U,S. Office of 
Education (1976, p. 12) stat* that, "States' reported Hispanic 
enrollments as 729,439' or 5 percent of the estimated student 
population of 14,238,471" that were enrolled in vocational education 
programs under their auspices. In the specific -case of Illinois 
of over 670,000. students in vocational education programs only 
21,629 are Hispanics, despite a population of nearly 1 million 
Hispanics. For a receM study funded' by the Illinois DAVTE (1980) 
it wy shown that 125 of a possible 750 local districts had been 
approved to claim LEP stucfents for FY 1980. Of these 127 approved 
One a.nd Five Year Plans, ,120 actually claimed reimbursement fc^r 
providing services to LEP students for the year 1979-80. The study, 
demonstrated that there were a diversity of services and programs 
available to LEP. students . This diversity of programs and services 
was shown when the 127 approved programs were reviewed for the" types 
of services provided to LEP students.- The co.ding for services was 
conducted according to the list of services for "Limited English- 
'Speaking" in the Illinois State Board of Education mimeo handout 
Occut^ational Education for Students with Special Needs . From this 
rev-iew process it became evident .according to Day and Phelps 
(forthcomijig) that:' 

there is an appareht lack of understar)ding of the importance 
of bilingual counseling as evidenced by the fact that only 
4 of the 125 approved plans provide such counseling. Secondly, 
there is also a lack of understanding of the importance of 
VESL as an effective tool for helping LEP students progress 
more rapidly both in English and In vocational education 

' V ■ 



It would appear that while many districts retogniz^ the 
need for ESL, few (4 Qf 125 to be specific) recognize 
the double-benefit-of VESL. (Day and Phelps, forthcoaiing) 

Only four, however, were identified as providing exceptional ♦ 

programming. Jhese exceptional programs specifically mentioned 

VESL and bilingual couliseling as part of their services. 

Day and Phelps (forthcoming) identify eight components that are 

critical in serviag LEP students in vocational education. The eight 

key components are: comprehensive planning, .curriculum (reflecting 

the culture and language of the students), staffing and staff 

development, assessment and testing, support services, physical 

facilities, scheduling and flexibility, and program evaluation 

plans. These areas if used as guideposts in developing programs and 

services will assure the implementation of a comprehensive vocational 

ducation program for LEP students. The recency of this concept of 

providing services and programs to LEP students requires adaptability 

and ^fl exibil i.ty on the part of vocational educators and administrators. 

In spite of the limited number of programs funded each year, there 

' can be no single "best*' program design, but is should be feasible 

to draw upon the successful experience of a number of programs. The 

process of evaluation may assist fICmders, administrators, instructors, 

students, and communtiy members 'to better grasp and understand the 

complexities of these new programs and seVvices. 

The Evaluation of Vocational Education Programs and Services Aimed 

at Limited English Proficient (LEP) Students . 

Evaluation is by it's nature a political activity. It 
serves decisionrniakers , results in reallocation of 
resources and legitimizes who gets what. It is 
intimately, implicated in the distribution of b^isic 
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goods in society. It is hiore than a statement of ideas: 
it is a social mechanism for distribution, one which 
aspires to institutional status. (House, 1976, p. 76) 

"'^^^^ Jho a bove quo^e by House depicts the process of evaluation as 
a "tool" in deciding who gets what. He goes on ?o state that 
"evaluation should not only be true, it should also be just". 
Justice needs to be an important standard by which evaluations 
should be judged. The evaluation of vocational education programs 
and services aimed at LEP populations in p^irticular requires this 
fair treatment. These serviced and programs aimed at LEP populations 
do not represent the typical vocational education .program. 

Although the literature relating to evaluation theory and 
,r^earch is much ricKer and more sophisticated today than it was a 
decade ago, actual results of evaluation studies themselves indicate 

^ that from an evaluation -perspective, many of the problems posed by 
the structural as well as the theoretical intricacies of the inno- 
vative programs remain unsolved. Guba (1969) commenting on the 
failure of traditional methods of evaluation states that " Inno- 
vations have persisted in education not because of the supporting 
evidence of evaluation but despite it". 

Invariably, decision-makers, educators, evaluators, parents, 
and the general public have to rational ize the persistent use of 
evaluation techniques with respect to the "justice" they deliver. 
The reasoning as to which evaluation technique or methodology to 
utilize in evaluating programs is a critical factor since the con- 
cept of justice limits the approaches one takes", what a9tivities 
one finds legitimate, what arguments count as significant (House, 
1-980), The ethics evaluators rely on will effect how they will go 
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about determining prognam effectiveness and thus future funding. ' 
The realization that the evaluation process is a political activity 
demands that an evaluation not only be true, but also be just 
(House, 1976), . ' ^ 

Bilingual vocational education rightly fits the description o^ 
what Cohen (1970) has defined as a "social action"- program. 
Bilingual vocatijDnal education meets the necessary criteria of 
"social action" programs; 1) it aims at improving the quality of 
education for the disadvantaged populations, 2) it is nojt aimed at 
improving conditions at one specific site, but rather at directed 
millions of children and adults, cand 3) it has been created and 
administered by federal and state government and not by any 
individual administratoV or educator. As specified by law, the 
aim of bilingual yocationaV^ducalion is three fold. Specifically, 
it should "^-A ? 

1, .provide the stajdent^with English and native language 

vocational instruction, * 

2, provide the student with vocational English as a second 
language instruction' while he/she is also learning an 
occupational skil 1 ; 

3, facilitate his/her adaptation and participation into 
the mainstream society, (Hurwitz, 1980) 

The first two aims are -directed at the fulfillment of the students 

Immediate educational needs and the third relied to the students 

long-term social goal": ' the attainment of a higher social and 

economtc status , 

» » 

This type of dual setting just described (i ,e, academic and 
social), is a characteristic which bilingual vocational programs 

20, * 



share with irany of other social action programs brought in under 
ESEA, The problems associated with the measurement of success of 
these programs continues to confound evaluators (Guba, 19691. 
Vocational education aimed at disadvantaged populations has- been 
particularly troublesome (KorTn 1980). It ha5 most of the problems 
common to non-ethnical ly oriented programs-spe.eifically those 
relating to 1) diversity, obscurity, and conflict within the goals 
of nation-wide prograijis, gind 2) the lack of clarity regarding 
measurement framework. Unlike other programs, however, vocational 
programs aimed at the LEP population confounds all the other 
iSifflcul ties* with still another reality - teaching and evaluating 
in two languages. 

^ Sanchez (1980> indicates that there fs a paucity of studies or 
evaluations concerning bilingual vocational instruction to LEP 
people. In a study conducted by Sanchez (1980) to analyze selected 
existing bil ingual 'vocational training programs within the western 
part of the United States, the data gathered indicated that^rogram 
evaluation addresses only outcome variables, i,e,, number of 

graduates, test grades, etc, which measured the students and 
♦ 

secondarily reflected program measures," (Sanchez, *1980, p. 197), 

The study r'ecommendecl the following: 

Planning for program assessment should being in the 
development stagehand continue into the implemental 
stage of the program, Jt should reflect student ^ 
olitcome variables, process variables, and organi- 
zational structure variables. Outcome variabl es 
relate to graduates, test grades, etc, wh41e 
process variables reflect such factors as teaching 
'Strategies, curriciilum planning, and mat^ial prep- 
aration, • Organizat\ional factors include examining f 
the administrative components of the program, 
(Sanchez, 1980, p, 213) \ 
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A two year study ef bilingual vocational training is currently 
being conducted by the Tnter-American Research As$ocia'tes, 
Specifically, the project is undertaking the study of nine selected 
bilingual vocational training programs in order to determine^ 

*^ 

program elements and strategies which have successfully contributed 
to the achievement of program gaa*ls. The nine bilingual vocational 
training/programs studied were the following: 

China Institute in America, -New York City, New York . ^ 

Chinatown Manpower institute. New York City, -New York 

Bronx Community College, Bronx, New Yow 

Miami-Dade Community College, Miami,* Florida 

SER/Hidalgo Jobs for Progress, San Juan, Texas 

£1 Paso Community College, EV Paso, Texas 

Little Wound School Board, Kyle, Soiith Dakota 

UCLA Dental Assistant Training Program, Los Angeles, California 

DeAnza Community College, Sunnyvale, California- *^ ' 
According toTroike (manuscript) the study identified ten criteria 
that would be considered in assessing program succe^ss,* The 
'Criteria serve as a touchstan^ for administrators aM" evaTuators 
who desire to invtiate or improve vocational training programs for 
LEP students. The cViteria identified are: 

!• Job placement rat,e. 

Quality of program planning, design, anci management, 

3, Competence and dedication of staff, 

4, Nature and appropriateness of instructiofl and curriculum, 

5, Appropriateness of occupation selected for training, 

22 




. ■ ■ f ■ 

6. Trainee r)&cruitment and selection, 

7. Behavior of trainees, including attendance, 

8. Learning <rate and achievemertt levels of trainees. 

9. Contextual changes: institutionalization, employer attitudes, 

- ---- 

i:0^. ~PrT)"gr"am"^rganl za t1 on and' mana gement . 



The Study of Evaluations: Meta-Evaluation 

Educational evaS4ation Is a relative newcomer to the field of 
education. Before 1965 tfiere were few evaluations of educational 
programs and also lit/le thought given to the necessity of this 
activity/ The evaluation rei|uirement in the Elementary and 
Secondary Education Act of 1965 and in other federal and state 
mandates resulted in a great deal of evaluation activity. At the 
present time, even though a great deal of evaluation activitiy is 
taking place, there have been relatively few people trained to 
conduct such work (Anderson & Ball", 1968). Furthermore, there 
exists today, a, widespread lack of agreement about wh^t evaluation 
is and how evaluation studies should be conducted (Guba, 1969i Wolf, 
1979). , 

__ ^According to Stufflebeam (-1978) , good evaiuatiofT requires that^ 



evaluation efforts themselves be evaluated. The evaluation of 
evaluatians is needed to both improve ongoing evaluation activities 
* and to assess the merits of completed evaluation efforts. The 
lable, "Meta-Evaluation" first used by Scriven T1969) , is typically 

^^sed to refer to evaluations of evaluation. Meta -evaluation \s a 
concept that is relatively in it*s infancy stafe. It is important 
not to forget, ho^vever, that meta-evaluation is new and is only now 

, . learning its mistakes and limitations. The recency of meta-evaluation 
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according to Cook and^Gruder (1978), must be kept in its proper 
'^^^erspective. . They state the following: 

... some persons might believe that evaluating evaluations 
by strictest technical standards is tantamount °to treating 
a developing science as though .it were' already a developed 
science with a large stock of accumulated wisdom. Since, 
evaluation reseaiP-chers caution each other not to evaluate 
new projects as though they were stable, it would hardly be 
logical to evaluate the developing art of evaluation in a. A. 
summative fashion which implied that it was highly developed. 
(Cook* and Gruder, 1978, p,. 13) 

The state of the ai\t of meta-evaluation is, at its best rudi- 
mentary and limited. It must be acknowledged that little can be done^ 
to overcome state of the'art problems other than* to urge 6valuaiorS 
/and educators to keep abreast of latest iiey^lopments in research 
Methodology and substantive theory. . » • 

Of particular interest to this study Bfe the actual meta-evalu- 
ations thk focused on the TPS. The.SACVE for Illinois sponsored a 
study^erftitl.ed The Efficiency and Efficacy of Evaluation Practices of 
. the Illinys Division of Vocational and Technical Education by^ Norton 
and Watley (1972). The main goal of the study was to determine the 
performance of- the TPS, The study .utilized several methodologies: 

1. Interviews of division personnel. . ' * 

2. Sampling of.agencies for study. 

3. Collection of data from record systems of the division. 
-4. Submissiffh 'of questionnaires to- fanners and team members. 
5. Independent study of trends in statewide evaluation, (Norton 

and Watley, 1972, p. 1-2). ' 

.The major finding of the study indicated' that the TPS was "well 

• ' \ - 

' received" by educators and noneducators . The study also concluded 
'■ that the TPS had several deficiencies, "most of which arise from the 
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fact that it does not emphasize or produce product information" (p. 2) 
Another meta-evaluation -of the on-site evaluation *phase oft 'the 

i 

TPS was conducted by Smith (1,979). The main purpose of the study was 

to provide "a basi^ for recommendations whic-ti could be. direj^ed toward 

the improvement of the on-stte evaluation phase" (p. 8). The study 

' employed the following methods: ' ^ ^ 

. 1. Two questionnaires were developed to obtain input from 
/ selected LEA personnel 'and on-site evaluatitpn team 

members after the second on'-site'visit . 



2. Selected data of record were obtained from DRVTE^to 
provide iJipasures of student and LEA pe/^onnel input 

'at the tiine 6t the first and second on-site evaluations. ' 

3. The data;obtairied from the^selected LEA personnel, 
on-site evaluation team members and the data of ^ 

- record were analyzed in response to the'research 
^ questions formulated for the study, (p. 29) ♦ 

The %fudy conc^luded that vocational programs that had recfeiVed 

on-site evaluations in 1970 and again in 1975, had to some.degre^ 

made "improvements in all occupational education components and that 

thejon-site eve^luation had "contributed to that improvement" (p. 219), 

In addition, both LEA personnel and team members reported that 

"Participation in the on-sit^ evaltjation was a benefit to themselves 

.and to their LEA occupational education ^gram" (p. 220). 

- . ^ ' ^ \ ^ . 

A study conducted by Cheaney (1981^>, focused on cieterming the 

'criteria an*^y5tems that were used to Identify handicapped and 

\ ^ ^ ^ " J * • 

disadvantaged students within the local education agencies in 



Illinois that operate secondary and/or post-secondary vocational 

education programs. The study did not focus an the effectiveness, 

of the evaluation procedures utulized'by the DAVTE, but did have 

. . V 

implications fqr vocational education efforts to identify and serve . 
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the handicapped and disadvantaged,' .Jhe stud^ was able to give spec^fi 
recommendations on the improvement of criiSfia aruK identification 
procedures that would enhance the^LEA's efforts to as^e'ss and pre- 
scribe needed services for handicapped and disadvantaged students, ^ 

fhesi^'two studies (Norton and Wat'ley, 1972; Smith, 1^79} which 
focused on the improvemeht of the TPS and the effett the TPS had on 
the improvement of vocational educat^ion programs*, did not address the 
concerns or needs of special need^ student^^^^5.^cationa1 education 
program^. Indeed, these two meta-evaluations .of the TPS have provided 
the basis for the actual improvement of the TPS and also in the\ 
pverall quality of vocational education programs in the State of 
Illinois. However, the continued ^improvement of the TPS and of 
vocational education programs throughout the State of Illinois needs 
to be placed into perspective with regard to the needs of LEP students 

^ As. stated earlier in. this chapter, thjs number of lEP students 
andHlieir needs have become increasingly recognized within the 1ast.>^^^ 
^^few years. In accordance with thrs recognition', more vocational 
education programs and ,serviceS" h^ve been aimed at LEP students , 
The continued growth, improvement and maintenance of these new 

» Vk t 

programs and services aimed at LEP students^ relies' hetfVily on the ' 
evaluation process, . 

The literature concerning meta-evaluation 'br evaluations '•'of the 
TPS do not reflect studies based on the case study methodology. 
Specifically, the literature revealed that meta-evaluat'ion models 
and methodologies that are either theoretically proposed or in 
actual use, are mostly based on empirical dat^^ and pre-established 
criteria and propositions. Meta-evaluations are to be conducted, ^ ^ 

26 ' 



according to the literature, with decis,ion-niake>s delineating what 
the areas of concern are, and what data needs to be gathered in 
order to make "wise and beneficial" decisions. 



A met a -evaluation of the TPS needs to be particularistic, 
holistic, and qualitative in nature* A case study with its bounda- 
ries defined. in terms of a: "study providing a better understanding 
of the process involved in evaluating vocational education programs 
and services aimed at LEP students in Illinois", may be classified 
as particularistic or focused. --The indepth focus on this tje(pic ^is 
justifiable on the basis 'that it is complex, sensitive and dynamic. 
A case study approach is viewed as a w^ to understand the com- 
plexity of a situation and its parts. Meta-evaluation models and^ 
methodologies do not reflect the case study methodology,^ As stated 
earlier, the state of the art of mefa-evaluation is, at its best, 
rudimentary and limited. The recent acceptance of the concept of 
meta-evaluation and of case study methodologies has limited their 
indivj^al and collective growth, 

__J , , . „ , . 
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• •/ CHAPTER III 

METHODS AND PROCEDURES 

OVERVIEW 

. The intgnt of the study was to examine the Illinois Department 
of Adult, Vocational and Technical Education's process of evaluating ' 
vocational education programs and services provided to LEP students. 
More specifically, the focus of the study was to provide a better 
understanding of the effects that^the TPS had upon programs and " 
services provided to LEP students. The study primarily utilized 
the case study methodology as a means of getting close to the 
phenomenon under 'study, ^ ^ 

The rationale- for utilizing the case study methodology stems ^ 
from the fact that it requires the evaluator using qualitative 
methods, to unders^:and the setting under study through direct 
personnel contact and experience with the particular program. This 
holjstic-inductiVe approach is based on perspectives developed in 
phenomenology (Bussis, et aT, 1973): The phenomology approach is 



concerned wrth understanding human behavior from each persons point 
of view (Spier, 1980), Phenomonelogist believe that human behavior 
cannot be understood without understanding the frameworkwrjhin 
which those being studied interpret their actions, feelings, thoughts, 
arid motives. 
Design of <the Study 

The design examined the effects of the TPS pn programs and 

% 

services aimed at LEP populations through the use of four procedures. 
The procedures were; 



28 



38 



1. Participant-observation, document aoslysis, and interviews - 
were conducted while the principal investigator participated 
^n Three on-site Visitations of local education agencies 
providing programs and services to LEP students. ' 

2. A questionnaire (See Appendix A) was developed to' obtain 
input from state vocational education dif^ectors, regarding 

J, what other states agencies ar$ doing in evaluating programs 
and services aimed at LEP students. , 

Inter\^iews were conducted to discover and verify the views 
and concerns of the DAVTE staff, local administrators, 
instructors, students and community niembers , 
4. A questionnaire (See Appendix B) was developed to obtain 
input from selected LEA personnel, regarding the impact on^ 
programs and services provided to LEP students that were 
evaluated by the TPS in Fiscal Year 1980. Follow-up phone , 
\ and personal interviews were conducted. In addition, 

document analyses were conducted to consider the composition 

c 

- ^ of the on-site evaluation team, recommendations made by 

» , ^ ^ 

the team, c[|inges in the One and Fiv^ Year Plan, and also 
the number <of LEP students being served by the LEAo 
The study collected and utilized both qualitative and--quanti- 
tative data. Multiple types of data were used in this case study. 
The use of multiple types of data to study a single problem or 
program is called triangulation accor4ing to Denzin (1978). This 
triangulation aids in correcting for the biases that are present i.n 
each type of data. Furthermore, triangulation acknowledges that 
evaluations usually have multiple purposes which must be carried 
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out under the most demanding of conditions' (l^eichardt and Cook, 1979) 

This variety of needs often requires a variety of methods, D^zin 

stresses the need for triangulation in the following manner:' * 

Because each method reveals different aspects of empirical 
reality, multiple methods of observations must be employed. 
This is termed triangulationX I now offer as* a finaK 
methodological- rule the principle that multiprle* methods 
should be us^d in every investigation, • (Denzin, 1978, 
. p- 28) • ^ 

The design was structured to acquire the infor^mation most needed 

and most useful in the given situatioji. This particular meta- 

evaluation of the TPS required comprehensive familiarization and 

understanding of the TPS, needs and concerns of LEP students, 
) t- • .... 
"^possible approaches in providing programs and services to LEP 

students, issues and concerns of detision-makers , ^and the socio-. 
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"political context in which tHese pr*ograms operate. . 
POPULATION AND SAMPLE 
On-Site Visitations 

— ' ■ — ^ 

The principal investigator participated in three on-site 

visitations (See Appendix which provides overview of the TPS) 
of LEA'S which were providing some progra^ns or. services to LEP 
students. The three sites were selected from a total population 
of 95 lea's which were scheduled to receive their on-site visitation 
(this takes place every five years) <. In fiscal year 1981, 13 of 
these 95 LEAs had been approved for claiming reimbursement for 
serving LEP^students , The three sites were selected from this 
sub-population of 13 LEAs, The selection was based on what 
Patton (1980, p,%154) has termed "purposeful sampling". In col- 
laboration with the pWject and the advisory committi^, project * V 
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staff, and the DAVTE stfiff, consensus was, reached concerning the 

three "typical" sites. The selection was based on the following 

V 

criteria: 

1. Must select at least one of each of the following ediication 
agencies: secondary school, area vocational center,' and 
xi post-secondary school . x 
^2. Must have a sizeable LEP population in community ^as verified 
by Regional Administrators, DAVTE staff, and advisory com- 
mittee members) . 
3. Must have been identified as an LEA currently providing 
■ pco^rams.and services to LEP students (as verified by 
analysis of the 13 LEA's One and' Five Year Plans for 
Vocational Education and also verified^'by the Regional 
Administrators, DAVTE staff, and project Adrisdry Committee 
members). 

Questionnaire on Evaluation Practices of Other States 

The questionnaire (See Appendix A) concerning the identification, 
'of evaluation practices being used to'^eet the evaluation require- 
ments ^setforth by Section 112 of Education Amendments Public Law 
94-482,^as developed, pilot-nested, and mailed to all fifj:y state 
directors of vocational education (plus seven territories). The ' 
purpose of the questionnaire was to identify the current practices 
of other state board of vocational education in evaluating programs 
serving LEP students. 
Interviews 

In-depth interviews were conducted to discover and verify the 
views and concerns ^of a variety of persons. The selection was based 
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on pre-fieldwork information collected at the planning stage of the 
study. This pre-fieldwork information was collected through the 

--sTeview of relevant literature, recommendations of the Project 
Advisory Committee and initial contacts and interviews with 
knowledgeable persons relative ±o the TPS. Interviews were conducted 
with the DAVTE staff, on-s,ite visitation team leaders and members, 
LEA administrators and instructors, and student and community 
members. Interviews began September 1, 1980 and were completed 
June. 22, 1981. It was assumed, that in view of the time constraints 
of the study, it was impossible to interview all persons who are 
involved in or who are affected by the TPS. However, based on 

" information collected on initial contacts, it was possible for the 
evaluator to be aware of the variations among the sites and person- 
nel . Patton in describing the problem of nepresentativeness states 
the fol lowi-ng: . 

By attempting to increase the diversity or variation in 
the sample, the evaluator will have, more confidence in 
those patterns that merge as tommon sites while at the . 
same time being able to describe some of the variation 
that has emerged to make programs unique as they adapt 
to different settings. (Patton, 1980, p. 102) 

The bottom line concerning the selection of interviews, according^to 

Patton is that "evaluator^ think through what cases they could learn 

the most from, and those are the cases that are selected for study" 

(1980, p. 101). 

Questionnaire on the Effects, of the TPS 

The initial population for this questionnaire were all LEAs that 
were either approved to claim reimbursement or actually did claim 
reimbursement for providing prograips and services to LEP students 



in Fiscal Year 1980. From this population, a subpopulation of LEAs 
identified that had received an on-site visitation in Fiscal Year 
1980. The 33 qualifying LEA's were divided i^ the following manner: 
LEAs that received an on-site evaluation and also were 
approved to claim reimbursement for serving LEP students = 27 
LEA'S that received an on-site evaluation and actually 
did claim reimbursement for serving LEP students = 6 
Total LEA'S w^re either approved or actually did claim 
reimbursement and also received an on-site evaluation in 
fiscal year 1980*^= 33 

^ The 33 LEA'S represented 26 secondary LEA's. Six post-secondary 
LEA'S and two'area vocational centers. The vocational 6ducation 
^directors of ^ch of these 33 LEAs received if questionnaire. After 
the questionnaires were returned and analayzed, three LEAs were 
identified for a further in-depth view. The selection fo these 
three LEA's was based on the following criteria.-' 

1. Representation of one Area Vocatonal Center, one post-secondary 
school and one secondary school. • 
V 2. A approval, by each LEA to participate in further in-depth 
interviews. 

^ 3. A sizeable LEP population exists in the community (as 

verified by Regional Administrators; local administrators. 
■ DAVTE staff and, individual Project Advisory Committee 
members). 

4. Identified as an LEA that in Fiscal Year 1980 and in Fiscal 
Year 1980 and in current Fiscal Year 1981, was providing 
some programs or services to LEP students (as verified by 

' 33 ■ • 



analysis of their One and Five Year Plans for Vocational 
Education and also verified by Regional Administrators, 
local aditiinistratoips, DAVTE staff, and individual Project 
Advisory tonunitte members) o 
As part of the further in-depth view, interviews were conducted 
with. the vocational education directors, personnel that dealt 
directly with LEP students (EoGo instructors, guidance or counseling 
staff), the on-site visitatldfn team leader., the regional adminis-* 

4 

trators and students. An average of four individuals were inter- 
viewed at 'each site^ 
INSTRUMENTATION AND DATA COLLECTION 

r 

On-Site Visitations and 'Interviews » ^ 

The interviews pertaining to the evaluation of services and 
programs aimed at LEP students in Illinois were- phenomenological 
in .nature^ Th6 phenomenological approach, according to Patton, 
is to b(5 used when the: ^ ' . ^ ^ ' 

^ interviewer wants to maintain maximum flexibil ity 'to be 
able to pursue information in whatever direction appears . 
to be appropriate, depending on the information that 
emerges from observing a partix:ula]f^ setting^ or from * 
talking to one or more individuals in tfiat setting. 
Most of the questions will ^low from the immediate 
context. Thus, the conversational interview is a major 
tool used in combination with participant observation 
to permit the evaluator who is participating in some 
programmatic abtivity to understand other participants* 

' reactions to what ii happening. (Patton, 1980,, p. 199). 

As Patton points out in-t|ie above quote, exact questions are at 
times not possiblejv how^yer, structuring the interviews around 
issues will assist thp researcher in discovering the foci qf the 
inquiry. An issue is a^ proposition about which people disagree. 
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according to Stake (1979), Stake describes it further: 

It is an idea about which there is an extra tension within 
. dynamic processes. It my be something that calls for 
watchfulness^ needing>»focus and attention. Whether or not 
'People are arguing about it is not cru.cial, but an issue is 
something that is felt deserving of and likely ultimately 
to be prominent in controversy, (Stake, 1979, p, 2), " 



^ Stake (1975) argues in another articl e that huma'ns are the 
best instruments for/many evaluation issues. The .important con- 
cern for the evaluator is to get his/her information in sufficient 
amount from numerous independent and credible sources so that it 



effectively represents the perceived std|:us of the program, 
however complex. Issues that were u^ed to structure the interviews 
were the following: , ^ ^ 

1, Overall strengths and weaknesses 'of the on-site visitation 
\ pmocess, - . 

2, Changes that might have occurred in programs and services 
aimed at LEP students as a result of the on-site evaluations 

3, Expertise and composition of team members on an on-site 
Ytsitation, 




Changes that might be made in the on-site visitation process 
with respect to the evaluation of programs and services 
aimed at LEP students. ' % ' ^ 

Identification ^f these issues and^ohcerns was conducted through 
several stages^, including^ initial interviews, initial data analysis, 
literature review, checking interpretation of interviews with the ' 
interviewers, and use of formal questionnaires,^ The interview data- 
were recorded using field notes. Since it^was ^humanly* impossible 
to write down eveny^tion that took place or that was said by 



35 



informants, field notes were taken in a condensed version. An effort 
was made to include direct quotes, phrases, single words, and 
/ unconnected sehtences/ After the actual interviews, an expansion 

0 f condensed . f i e ld notes was a ccomplished . I deas t hat seemed to 
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be issues or concerns of major importance to the resg^ondents were 
numbered sequentially and related topics were also coded with their 
main-issue number. ^ 
Questionnaire on Eval uati on Practices of Other States 

, The questionnaire which was mailed to state vocational education 
directors was designed to provide some empi ricaV account of how many 
states have a separate evaluation process for evaluating vocational 
education programs serving LEP students • More specifically, the 
intent of the questfonnarre was to identify exemplary evaluation 
, practices in other states. 

The questionnaire was reviewed by the Projejtjt Advisory Com- 
mittee and was pilot-tested -and reviewed by one on-site team leader 
and the Head of the ProgVam Approval and Evaluation Section of the 
DAVTE. Data collebtion of this questionnaire began on November 
t 10, 198Q and continued until April 1, 1981. * The time schedule for 
* data collection involved the initial mailing on November 10, ^980 
and a folloW-up mailing one month af*ter the initial mailing. 
Those state directors who did not respond to the initial mailing^r 
to 'the fqllow-up letter, received a follow-up phone call between 

« 

^^anuary 19 to January^ SOi 1981. The phone call reminded the 
respondents' of the nature of the study and the importance of their 
participation in the study* 
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Items included in the questionnaire were the following: 

— Does your state have a»separate evaluation process for 
evaluating vocatfonal programs serving LEP students? 

Rate this statement; Your state is effective in meeting 
t-he-requirements setforth by Section 112 (Title II, Edu- 



cation Amendments of 1976) in evaluating all vocational 
programs , 

— Rate this statement; Your state is effective in evalu- 
ating services and programs aimed at lEP students. 

— Rate this statement; Your state is effective in utilizing 
evaluation results in improvement of services and programs 
aimed at LEP students . 

— Rate this statement; The ^evaluation efforts in your state 
nejed to be revised to meet the needs of LEP students. 

— Do you feel your state's follow-up studies of LEP students 
need to be improved? If yes^ why? 

-- Do you feel your state's employer follow-up studies of 
LEP students need to be improved? If yes, why? 

Questionnaire on^ th e^ Effects of t he T.P.S. 

The questionnaire which was mailed to 33 LEA vocatlanal 

education directors was designed to provide some quantifiable 

measures*of the effect the on-site visitation had on programs and 

services provided to LEP students. This questionnaire was developed 

with the guidance of a questionnaire which had been previously used 

to assess the TPS (Smith, 1979). The questionnaire used by Smith, ~ 

was utilized to determine whetlaer:^ny changes in vocational education 

Rrograms could be related to thfi^oft-site evaluation visits which 

had occurred twice within a five year period. 

The questionnaire was reviewed by the Project Advisory Commit- 

\k, tee and the DAVJE staff • The questionnaire was also pilot-tested 

3^ith four lea's. Vocational directors of these four LEA's were 

ajsked to pilot-test the questionnaire to determine the suitability 

) 
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of the format (Did the format cause confusion? Was it readily 
underst;pbd?) and iten\ charact^istics (Were items easy or too 
hard? Were items all answered?) o Information gained through the 
review of the questionnaire and the pilot-testing wa>utilized In 
developing the final copy of the questionnaire. 

The time schedule for the col lection involved an initial 
mailing of April 15, 1981 and two follow-up phone calls at two 
week intervals. The follow-up phone calls attemp'ted to determine 
whether the questionnaire had actually been received by the respond 
ents. (In three instances, respondents stated that they had not 
received the questionnaire. These three respondents were sent /a 

follow-up' letter, a second copy of the questionnaire and ajiother 

f - ■ ' 

return addressed, stamped envelope). The focus of the follow-up 
phone calls centered on the nature of the study and the importance 
of the'ir participation In the study* Some of the items Included in 
e questionnaire are the following: ^ 

Rate cultural sensitivity to LEP students by vocational 

^ • / 1 

education 1nstni£;feors.^*^ ' * 

Rate cultural sensitivity to LEP students by guidance 

' and counseling personrteK 

Rate staff development concerning is^es and concerns 

of LEP students In vocational eflUcatlon programs and, 

; . services. * . 

-- Rate the statement; To the best of my knowledge. the 

OAVTE evaluation team members'who interviev^d our LEA 

•V 

were knowledgeable in the area of LEP students. 

" • ' ■ ^ .• , ' . • V 

.— Rate' the statement; ■ Through' thfe on-site evaluation, 

f 
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our^LEA became'aware of resources and services that will/'^ 
\ are being used to improve programs and service9%for LEP 

students. - . 

— Rate the statement; Many of the changes that will/are 
being made in our vocational education programs and • 
services for LEP students have been the result of infor- 
• mation provided by the on-site evaluation report. 
DATA ANALYSIS . ' 

'\ 

Questionnaires • . . 

« ^* 

The analyses of both questionnaires, required for the purpose 
of this study, included th^^mputatlon of descriptjve statistics. 
Descriptive statistics were used for summarT^ing data systemati- 
cally for ease of. comprehension.. Attempts to make broad generali- 
zations and Interpretations are not to be attempted with the>data, 
^he empirical measurement of key aspects related to the study wlifl 
assUt th.e principal evaluator In gaining a different persp'feclrh 
which will cqmptement the qualitative data collected, Thte empirical 
'data aided 1ti the cfiscov.ery and verification task of the^prlnclpal 
Investigator. 

bn-Slte Visitations and Interviews 

According to Splrer, the phenomenological approach to data 

analysis is continuous : . \ 

Analysis of case study data is an ongoing process that 
begins as soon as the first^ piece of datum is .collected, 

if This featul^ 1,e,, '^analys'is as you go," distinguishes 
the case stupy form other methodologies .in which data , 
collection and data analysis are discrete activities, 

* (Spirer, 1980, p, 61) , i 
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As data was collected and immediately analyzed, inferences were 

drawn, n^w questions raised, and themes develpped which adjusted the 

• focus' and schedule of the interviews and observattaTTS~tSptrer,^980)T 

Data analysis involved the making of patterns, themes, and categories 

for the data. The divisions^ emerged put of the data rather than 

that of imposed divisions placed on them prior to/data collection, 

%iH^pr6t^s for constructing the case sjtudy is the process 

proposed by Pat ton: 

3tep One: Assemble .the raw data • 

' This data consist of all the informatton collected 
about the personi or -program for which a case study 
is to be written, _ 

Step Two: Construct a case-record. 

This is a- condensation of the raw case data 

organizing, classifying, and editing tfie raw 

case data ir\to a manageable and accessible palckage, , 

-StepThree\ Write a case Study narrative, 
, ^ the case studyjis a readable, descriptive picture , 

W'^ . ; /.of a person or^.ogram making accessible to the 

reader all the i-nforiktion necessary to understand 
the person or progra^'m,!. The case stud/ presents a 
'holistic portrayal of a person or a program, 
^^<^\ MPatton, 1980, p. 304} 
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CHAPTER IV 
ANALYSIS AND FINDINGS 



* OVERVIEW . — _ - 

The purpose of this study was to provide a better understanding 
of the processes used by the DAVTE in evaluating vocational education 
programs and services designed to serve limited English proficiency 
^ students in Illinois. Specifically, the study focused on the TPS's 

effefcts iipon vocational' education programs serving iEP students in, 
. the state of Illinois. This chapter presents the results of this 

^ study. Ike organization and presentation of the data wiy follow 

the major four research questions for the study: 
^ 1. What procedures has the DAVTt undertaken in order to 

evaluate programs and services aimed at LEP populations?- 
2. What are other states doi-f^ evaluating services and 
programs aimed at tEP populations? 
..3. What are the views and concerns of the DAVtE staff, local 
^ ' • administrators and board members, instructors, students 

and community members with respect to the evaluation of - 
^ ' programs and services aimed at LEP, populations? 

4. What is the extent and nature of the impact on local 
- education agencies that were evaluated in FY 1980 and are 
serving LEP students? 
~ EVALUATION PROCEDURES OF THE ILLINOIS DAVTE ' 
^ The Illinois State Bo^rd of Education (DAVTE) developed and 
■ Implemented aVformal planning and evaluation system to insure the 
maintenance growth and quality of aJJ_ vocational education programs 



41 



and servii:es'. In order to fully understand the procedures of the 
TPS several types of data were collected. Data were collected 
through participant-observation in three actual on-site visitations 
and in. the-Team Leader Orientatiori Session held in early Fall of 

1980, and also in the Team Leader Wrap-up held in late Spring af 

1981. Data were also collected through the review of significant 
documents. Documents reviewed were the following: 

1. ^ Rules and Regulations for the Administration of Vocational 

Programs Illinois State Board of Education. 

2. Vocational Education Data System Report for 1979-80;' 
Illinois State Board ucation. 

3. Guidelines, and Format for^Local District One and Five ' 
Year Plan for Vocational Education; Illinois State Board 
of Education, FY 1980-81 . 

4. Team Member Handbook; Illinois State Board of Education, 
1980-81. 

5. Team Leader Handbook; Illinois State Board of Education, 
FY 1980-81 

6. Evaluation data gathering instruments (See Appendix D) 



Ao Student^a Faculty, Preliminary Evaluation Instrument. 
B. School and Community Data Fgrm. 
Co Team Leader Questionnaire. 
^ D. Student and Employer Fol low-Up. 
In addition to the participant-observation and document review, 
data were also collected via interviews with team members, team 
leaders and staff from the DAVTEo' There interviews were extremely 
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helbful in better understanding the intricate and complex segments 

of the ^'entire TPS. 

As a result of reviewing documents related to the TPS, it is 

clear that £he intent of. the systenTis" quite Troad in scoped The 

comprehensiveness of -the TPS is described in Appendix C. V Readers 

are encouraged to read the material in-^Appendix C, (which describes 

the current 1980-81 TPS)^ prior to reading further, 

Evalua'tion Data Peffcning to LEP Students in thgTPS 

In reviewing these instruments, it is evident that some data 

is collected with respect to the eyalyation of' programs and services 

aimed at LEP studetns* The Faculty PEI accumulates more specific 

data concerni^ng LEP students than all of the other instruments. 

The Faculty PEI has a total of 27 respon^ items. Of these 27 items 

four items provide specific data VegSrdi^g LEP programs and services 

The four items are the following: 

Item 14, Does your Agency have a system for identifying 
disadvantaged, handicapped and limited English 
Proficiency students? 



Yes 



No 



Item 15. Do 'you have students in your class{es) who ai'e: 
a. Disadvantaged? 

^Yes No 



b. Limited in E/iglish proficiency? 

res No 



Uncertain 



Uncertain 



c. Handicapped? 



^es 



No 



Uncertain 



Item 16. Are additional sWvices provided by your agency for 
disadvantaged, limited English proficiency and ~~ 
handicapped students (other than special education)? 



a. Disadvantaged? * 

Y es ^No ' ^Uncertain 

b. Limited in English proficiency? 



^es No Uncertain 



c. Handicapped?. ' ■ ' 

^Yes ^No Uncertain 

Item 26, Have you participated in vocational in-service . , 
activities which emphasized equal educational 
opportunity .for all students? 

Yes /- No 



These four Faculty PEI items are used as a ma|ar source of data in 
.addressing the Tasks in the Team Member Handbook which address LEP 
issues. The Student PEI does not provide any information relevant 
to this issue. However, one must consider if the student PEI could 
be completed at all by students with a limited English proficiency 
ability. 

The- Team Member Handbook provides the team members with three 

specific Interview Tasks related to LEP programs and services. The 

first interview task is found inf the Student Services section: 

Task. 4, Determine the appropriateness of the criteria used 
to identify di^dvantaged , handicapped and limited 
EpflUsh proficiency students and the impact of 
additional services provided to these stadents, 

(PEI Faculty 14)> 

(PEI Faculty 15) . \ ' * 

(PEI Faculty 16) ) 

The next two interview tasks are itientified as "Access and 

Equit^" Tasks under the Program Management Section: " 

Tasl< 1. Determine the adequacy of ajgency efforts to assure • 
- access for all students, including those with special 
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needs, to all vocational education programs and s^vices 

(PEI Faculty 15) 
^ (PEI Faculty 16) * ' 

Task 2, Determine if the agency's vocational programs and 

student services are free of sex bias/sex role steret 
typing arid culturaj di-fferences . 

' (PEI Faculty 26) • " 

The School and Community Data Form which is filled out by the 

fand Five Year Plan writer has one item (which is identical 
or interview taslcjust presented) « Both^f these items, 
lar, ask if vocational^rograms and services are free df 
ifferences. Hqwever, in providing. programs and services 
dents, cultural differences many times could be encouraged 
hasis placed on sensitivity by faculty, and students toward 
cross-cultural differences. 

The Student Proc^ing Unit" Record (SPUR) is a data form used 
by LEA's to record individual student background information along^ 
with the results of the student and employer follow-up surveys. 
LEiJi's are required to complete a SPUR for every studentvthat has 
received vocational education training and has completed or" left the 
progjram. LEA's compile the SPUR's^for alTstudents and submit them 
to the DAVTEo The DAVTE utilizes the SPUR data to produce Team 
Leader Follow-up Reports for LEA's that are .being evaluated. 

'The SPUR includes two items from the student's -background 
which provide vital, information concerning the success of LEP 
students. The items indicate the students racial/ethnic designation 
and also classifies special needs students into three categories: 
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(1) Handicapped, (2) LimUed English Speaking, and (3) Disadvantaged. 
The Team Leader Follow-up Reports, however, do not present data that 
reflect student racial/ethnic background or, special needs classifi- 
cation. This type of data is not provided to the t^eam members or 
Team Leaders in any other format or content, unless specifically 
discussed in interviews with LEA personnel . 

Data from Participation -Observation in Three On-Site Evaluations 
^ ~ 

Information gained through participating' in three on-site 
visitation^, provided an opportunity to discov^^ffjid verify different 
themes, issues and concerns relevant to evaluating LEP programs and 
services. This activity provided a natural setting to gain direct 
experience in evaluating all aspects of vocational education programs 
The complexities of evaluating an entire LEA program became vividly 
apparent. The role of the principal investigator durjng three 
visitations was to actively participate in the evaluation process 
and to observe and collect data on ,the actual process itself. The- 
team members and leader were made aware of the principal investi- 
^gator's rpl-e prior to the start of the on-site evaluation. An 
attempt was made to collect insights in a natural setting while 
collectijig data in an open overt miiQner. 

In 'the majority of instances team members, leaders, and LEA 
personnel appeared to be* at ease and comfortable with the partici- 
pation of the principal investigat^or. Opinions, views, and concerns 
were readily furnished by participants offthe on-site visitation. 
Most persons were eager to ask questions or^ive statements of the 
following character: 
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"What type of services should be provided to this kind (LEP) 
of students?" (Team Member) / 

# 

"In situations where you have a group of Indo-Chinese just drop 
into your school, hov/ are you suppose to provide vocational 
training if they cannot speak any English at all?" (Team 
'Member) 

"I*m a counselor at my* school (Secondary), but I would not 
know what type of questions to ask of counseling departments, 
to see if they were meeting the needs of LEP students, I 
' suppose, I do not know what to look for," (Team Member) 

The three on-site visitations were conducted at distinctly 
different LEA's, Each LEA provided, within their own context ^ 
distinct approaches in meeting the needs of LEP students, A 
description of each of the LEA*s will follow. 

The LEA which provided the most diverse programs and services 
to LEP students was an urban conmlntiy college with an enrollment 
of 5,637 students in the Fall, 1980, The student body reflected ^ 
number of ethnic groups. The neighborhood of this community re- 
flects the^ most ethnically diverse center of urban poverty in- the 
United States according to the LEA's One and Five Year Plan for 
Vocational Education, The ethnic background of the student popu- 
latlon^for the Fall of 1980 was as follows:' 



Ethnic Background 


No . 


_1 


Asian 


1,124 , 


19,9 


Native American 


149 


^.6 


Black- 


1,079 


19,1 


Hispanic 


• 836 ' 


' 14.8 


White 


1,969 


34.9 


Refuse to Indicate 


48Q 


8.5 


Total 


5,637 


99.8 
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This uirban xommunity college along with it's multi -ethnic 
neighborhoods provided a multiplicity of programs and services to 
meet their needs. These include: 

1. English As A Second Language Program, % 

2. Tutorial Project . 

3. Language Skills Center. 

4. Mastery Learning in Courses and Disciplines. 

5. Mastery Learning Retention Project. 

6. Adult Pre-EntrJr/rogram. 

7. Plato Center (computer assisted learning center). 

8. Vocational English As A Second Language. 

9. Bilingual Vocational Center. 

10. Native American Vocational Training Program. 

11. Russian Refugee Program. ^ 

12. Indochinese Refugee Program. 

The team leader for this evaluation was a Dean at a suburban 
community college. The Team Leader had extensive experience in 
vocational education and in prior evaluation work. In discussing 
the needs of LEP students with the Team Leader it was evident she 
held a clear understanding of LEP students needs and how to meet 
those fleeds. The team members as a group appeared to reflect a 
sensitivity to LEP student needs, but some seemed unaware of 
appropriate strategies to meet those needs. 

The evaluation report contained four conclusions with 
particular impact on LEP student programs or services. They were 
the following": 
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Conclusion: 

1.5^ Based on interviews with administrators; ^college is 

to be conwended for placing the directors of the following 
programs in the regular callege budget: Bilingual \^Qx:a- 
tional Program, Russian Refugee Program, and thejutorial 
Assistance Program. 
Recommendation: 

!• Continue and expand institutionalization of other programs 
serving special needs students.. 
Conclusion: 

2, Based upon the. faculty PEI and interviews with members of 
the college community, the criteria used to identify disad- 
vantaged, handicapped, and limited English proficiency 
students and special services to assist these students are 
widely known and utilized. 
Recommendatiorr: 

2, No recommendation. 
ConcVusion:' ■ * 

3. Based on the student. and faculty PEI data, and interviews 
with administrators,, faculty, and students, cojlege 

• is to be commended for meeting the heeds of the LEP student 

through various approaches Including: ESL, VESL, Tutorial 
Assistance, language skills instruction, Plato instruction. 
Bilingual Vocational Program, Native-American Program, _ 
Russian Refugee Program and the Indochinese Refugee Program 

Recommendation: 

3. 'Continue^these fine efforts and expand as appropriate. 



Conclusion: 

4, -Based on'the review^of t7)TlJrvr--aiTti^Five Year Plan* Update, 
^ " and Interviews 'With adminis'trators, the Bilingual Vocational 
Center is a worthy and commendable program which meets the 

needs of LEP students at ^ college. The program is 

4* being supported and assisted in its development by adminis- 
• -trators, faculty and community members. 
Recommendation: 

4, Continue and expand the worthy a(nd commendable program. 
Suggested Improveffients : 'v^^ / ^ 

4a, Consider renaming the program to reflect its actual 
bilingual support services and not simply bilingual 
instruction, A possible name could be "The Bilingual 

r 

Assistance Vocational Center". 

b. Consider expanding offerings to other vocational areas, 

c. Consider hiring one full-time job developer, 

d. Consider expanding enrollment to more LEP students, , 

e. ' Expand the communication of availability of support ^ 

services to all eligible students. 
The suburban community college had an enrollment of 6,000 
students in the Fall of 1980, The college itself is located in a 
small rural town with an approximate population of 1,500 persons. 
However, the community college serves a Chicago suburban community 
of approximately 90,000, Within|this context, information gained 
through interviews suggested that approximately 20 to 22% of this 
population is Hispanic, and about 1% is Black, 

50 



The services' provided efccording to their One and Five Year Plan 

include an English as a Second Language program, a Vocational 

English as a Second Language program funded under a federal grant, 

/ 

and a project designed to develop and compare two vocational edu- 
cation programs for LEP students, funded under- the DAVTE (Research • 
and Development Section). ' ' * 

The team leader for this evaluation held a position as an 
assoQiate professor at a major state university. He had extensive 
experience in conducting vocational education research and evaluation 
4f vocational education programs. Through many years of partici- 
pating as a team leader in the TPS, this team leader had a indepth 
historial perspective into the development of the TPS. This past 
experience with the TPS clearly gave him a sensitivity to^LEP 
student ^needs and also in meeting the needs. In additioTi to this, 
*he appreciated the difficulty in attempting "rt^^^e stated: 
"How can you evaluate these (LE*P) programs whej) the system (TPS) 
was not designed to evaluate in such detail". 

The team members on this on-site visitation Jid have members 
which were sensitive to LEP needs,, but were not .experienced 
in dealing with or providing services to LEP students. The evalu- 
ation report contained three conclusions with particular impact 
on LEP student programs or services. They were the following: 
Conclusion: 

1. 'Based upon interviews with faculty and administrators, it 
is evident that remedial reading, writing and math courses 
. are available to handicapped, disadvantaged and Limited 
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English Proficiency (LEP) students. Rehabilitation 

Services (RSA),. Hearing Impaired Program, (HIP) , 

ESL, and bilingual vocational program student benefit 
from a comprehensive range of services which should be 
available and^f rticulated to other handicapped, disad- 
vantaged, and, LEP students who may need them. 
Recommendation: 

Continue to provide special services and expand them ^r other 
students with special needs. ^ 
Suggested Improvements: ' 

la. Continue to provfde supportive services which are currently 
available. . 

b. Communicate to handicapped, disadvantaged and LEP students 
not enrolled in special programs, the availability of ■ 

.existing support services 

c. Asse^ss the need, for additional services. 

d. Provide additional support services- on an "as needed)* basis,^ 
Conclusion': 

2. Criteria for identifying handicapped, disadyefttaged, and 
limited English proficiency students, is evident in the 
One and. Five Year Plan. However, no central clearinghouse 
is currently available to deliver support services to 
those students who may require them. 
Recommendation: 

2.^ Develop a means of coordinating the delivery of special 
services -to students in need^ ' - 
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Suggested Improvementrs: ^ 
2a. Conduct a systematic follow-up in reg.ard to the delivery 
of support services to those students identified as LEP, 
handicappecj and/or disadvantaged, 
^bv^rovide appropriate support services for each student 
•identified. ^ - 

c. Assign a staff member responsibility for coordinating.tfve 
• delivery of the support services^^ 

d. Commiinicate the availab^il ity of support services to all 
students. . . 

Conclusion: ' ' • 

3. According to^ observations and interviews with administrators 
faculty, and community members, the LEP student vocational 
program is, a worthy and commendcible approach by the college 
to meet the needs of LEP students. The. program is being 
supported and assisted in its development by administrators, 
faculty and community members. 
Recommendation: - * . ^ 

3. Continue the \^ student vocational program. 
Suggested Improvements: ..^ ^ 

3a. Discuss ways to keep this-^program ongcJ^^ng: 

b. Expand the number of programs' aVailfSlB^to LEP students; 

c. Increase the'number.of students enrolled in this type of 
program. " , 

The suburban secondary-school is part pf one of the largest 
school districts in the state of Illinois. There are eight 
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secondary schools within this district. The district serves the 
''northwest suburbs of Chicago, According to interviews with the 
LEA personnel, a sizeable LEP population, does not exist in the 
district. However, interviews with community members .reflected 
that there was a sizeable increase of Hispanics, European immigrants 
and Indochinese immigrants. 

The services provided to LEP students- according to their One 
and Five Year Plan infclude a building speech therapist who works 



with students one on one,^nd a bilingual program cfesigne^^'to help 
who DC 
r^am 1 



students who oossess limited English speaking abilities. 




The tteam leader evaluating this seco"hdary s^chool held a 
^ ^ position as an assistant professor -a^ a| major statje university. 
She had^ extensive experience in dissemil^ati ng information on \\m 
/ to better^sprve special needs .students in vocational education and 
was currently participating in her 2nd year as a Team^ Leader, 
Professional experience inducted directorship of a proafct which 
identified exemplary vocationa^l education programs and services 
meeting the needs of handicapped and disadvantaged students. In 
additibn^o this, she was actively involved in numerous dissemination 
and in-service activities focusing on how to better serve special > • 
needs students in vocational education programs, ^ * 

Various team members as in the previous on-site visitations 
were sensitive to LEP student needs but were not experienced in 
dealing with or in providing services to them. The evaluation 
/eport contained two conclusions with-^particular impact LEP 
students programs and services. They were the following: 

s 
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CbncTusi'dn: 

• !• It was evident that 



High School does provide 



additional services through the following programs: 
EMH, LE, IR and WECEP. However, it was found that* 
— : vocational instructors need -to modify program objectives, 

te'aching methods, and materials to meet the individual 

\ 

needs of handicapped and disadvantaged students • 
Recommendatidn: 

1. Efforts should be initiated to encourage vocational 

instructors to modify and supplement program objectives, 
\^ ^ teaching methods .and malerials to meet the individual ' 

needs of special' needs students. 
Suggested Improvements 

^ la. Provide in-service training for vicational instructors, 
"•on how to modify and supplemejit program objectives, 
teaching methods and materials to better meet the nee^ 
of speci^il needs students, 

b. Utilize consultants to assist vocational instructors on 
a regular basis, 

c. Utilize State Board special needs consultants anrd materials 
relevant to this topic. 

d. Establish a proced^ or mechanism to assure that the * 
needs of the students are being met. 

e. Evaluate ongoing efforts, and generate reports for futiN;;e 
services needs. 
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Conclusion: 



2. -Based on interviews with instructors guidance staff » 

-administrators and students, it was evident that limited 

English profii;4^cy students are not participating fully 

in vocational education programs and services, 

Reconmier^atlon: ~ * ^ 

* 

2. Efforts should be Initiated to encourage 11mited)Engl1sh^ 

♦ 

proficiency students to participate in vocational education 
programs when appropriate. 
Suggested Improvements: 

2a. Consider the establishment of a bilingual vocational 
education program. 

b. Utilize State Board staff and materials concerning these ^ 
•students . ~ , • 

c. .Contact the Bilingual Vocational Education Project in 
/ Arlington Heights for assistance In id^entifying possible 

Iternat1ve5 in serving LEP students, 

d. D^>m1ne the" feasibility of establishing a Vocational 
English asVa."Second Language (VESL) program. 

e. Encourage collaboration between vocational Instructor? 
and bilingual edtfcation instructors to develpp strategies 
on how to modify and adapt program objectives, teaching 
methods and materials to meet the needs of LEP students, 

f. RecruTt a Hispanic community member.on the advisory council, 

In retrospect, the opportunity to participate and observe , 
three separate on-site visitations provided much me^miTi^fu^l ^ 
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information. Probably the most powe&ful observation was the reactio 

of ^ the team members to the entire process. The positive feelings 

regarding the rapid setting bonds of teamwork, cohesi venBSslmd 

cooperation were common in the three teams. Team members sensed the 

entire activity as being useful both for the LEA being evaluated and 

for ^!^selves. Ihe^r perspectives on the final evaluation report 

were of the following nature: ^ 

".The report has addressed every aspect relating to voc. ed., 
it's amazing how in two days we could haye learneS so much 
about this school" (team member) 

"It (^e final report) is comprehensive and more important, 
it's going to be re'ad. When our school was^ eval uated, all 
of us (instructors) got a copy." (team member) 

Team members also expres^-^views concerning the benefits for 
the team members t+iemselA/es , . The opportunity ta .see what other 
LEA'S were doing in their reTat€4^ fields, and also the opportunity 
to exchange ideas among other team members and LEA personnel was 
viewed as a "terrific oppart;unity to grow.and meet new people" 
(team memteer.) . In addition, they viewed the experience as helpful 
in preparing for and gearing-up, for their own LEA evaluations. 
The opportunity to become involved in an actual evaluation and to ' 
know, In advance what "evaluators" are focusing^^, was considered 
a distinct advantage if their LEA was scheduled for an on-site 
visitation in the near future. 

In the process of the on-site visitations, team members 
expressed a quandary as to what to look for, or what to ask LEA 
personnel in relation to LEP programs and services. An interview 
task under the Student Services section in the Team Member Handbook! 
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specifically focuses on the appropriateness -of identification 
criteria and the impact additional services to LEP students. 
However, little information is given in the PEI's and other 
^evaluation instruments.* A team member expressed the following: 
"The interview task is specific enough, "but how do we (team members) 
know what is appropriate criteria or what is an appropriate or good 
impact". Most team members which had LEP students in their own 
LEAvN^^noted that there seemed to be a clear shortage of information 
on "how\^" serve LEP students in vocational education. 

Wheivieam members were asked the following question: "Do you 
feel the d^n-site evaluation process nee.ds to be changed with respect 
. to the evaluation of programs and services aimed at LEP students?", 
the typical response was: "There probably needs to be more emphasis 
placed on LEP students, but I wouldn't know what exactly to change" 
'(team member). Very few structural or process changes were recom- 
. mended by team members. The general perception observed was that ^ 
team members felt uncomfortable evaluating such programs or services^ 
' based on their past professional experience which general ly appears 
to 6e very limited relative to programs and services for LEP students 
^ in vocational education. 

The roleof the principal investigator as a team member and a 
data collector for a separate 'study was a difficult task; Team 
leaders and members identified the principal' investigator as an 
"expert" in LEP student^. In line with this -"exp^i^se" came 
^ pijimary responsibility of writting conclusions, recommendations, and 
suggested improvements dealing with such topics as s?ucjent services. 



58 



and special needs dent issues. In addition to this, a substantial 
amount of informal inservice activity took place with fellow team 
members and LEP personnel that were interviewed Vegarding^ow to 
identify and serve LEP students. These activities took place in the 

midst of attempting to gain a true holistic view of the entire ^ 

.A 

vocational edugatiqn program. 

. ft 

EVALUTION PROCEDURES OF OTHER STATES 

Questionnaires were sent to state vocational education directors 
of forty-nine states (excluding Illinois) and the folJowLng United 
state districts or territories: District of Columbia, Guam, Puerto 
Rico, Mariana Islands, North Mariana Islands, Samoa, and the Virgin 
Islands, Data were obtained from forty-one states and four terri- 
tories which represented a total of 45* responses. Additional data 
were also obtained from state education agency (SEA) personnel"* by 
a' follow-up phone interview. 

Of the forty-five responses, three SEA's did return the question 
naire but d^jd not complete it. Two of these three no' response 
questionnaires explained their no response by this type of statt^ment: 
"We have-uo programs for LEP students, therefore, no evaluation 
process" (State- Director of Vocational Education). The other no 
response questionnaire stated that the entire evaluation system was 
being. revised and therefore did riot feel participation in the study 
would be helpful . 

In order to determine whether- states had a separate ev^aluation 
process for evaluating vocational education programs serving, LEP 
Students (separate from rei^lar vocational educartion evaluation 
or other state evaluation systems) respojidents were 'asked to , 
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indicate positively if they did have a separate process or negatively 

if they did not. If the answer was YES, repondents were asked to 

provide a brief description of their evaluation process. The 

responses were as follows: 

Zy States responding "YES" 1 (2.2%) 
States' responding "NO" 41 (91.1%) 
No Response 3 (6.6%) 

It should be noted thaf the one response which answered YES, 

did provfde a brief description:- 

Projects funded with LEP funds must submit a Disadvantaged 
project application if the project is for instruction. 
These applications specify an evaluation plan separate 
from regular program review measures. 

The-data indicated that 91.11 percent^of the respondents did 

not have a separate evaluation process. Some respondents did pro- 

vide explanations fdr their response: 

We are interested in. learning from other states how they are 
coping with the influx of LEP people'who need vocational 
training, I would like to prefer to accept the opprobrium 
andT request your help. Does anyone have program guidelines, ^ 
standards, criteria for evaluation for vocatiojial programs 
serving LEP (State Program Consultant for Special Needs 
Program^. 

Our state at present has 'no specifically identified vocational 
programs for LEP students. This is limited to the limited ' 
^population scattered throughout the state (Assistant Director, 
Division of Vocational and Technical Education) o 

In , we oniy^have one LESA (limited English Speaking 

ability) project operational at this time (State Coordinator 
of Vocational Education) o 

We have not had a request for funds and efforts to seek LEP 
Clients enrolled in vocational education produces nothing 
(State Director of Vocational Education). 
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- ^ Respondents were asked to agree or disagree to four questions 
in the questionnaire. Table, 1 summari2es information regarding the 
respondents perception concerning the effectiveness of their own 
evaluation process. 

More than 80%>espondents indicated an "agreement" or "strong o 
agreement", in the effectiveness of their state in meeting the 
requirements set forth by Section 112 in evaluating a_n vocational 
education programs. However, a considerable decrease in this 
feeling is noted, (46;6%) in assessing the effectiveness of their 
state in evaluating services and programs aimed at LEP students. 
OveApn (37.7%) of the respondents were neutral with 

respe^ to effectively evaluating LEP programs and services. 

Exactly 33.33% of^tjTe respondents agreed or strongly agreed 
that their state was effective in utilizing evaluation results in 
improvement of services and programs aimed at LEP students. .However, 
the same question elicited a neutral response of 44.4%. 

Thirty-eight percent (38%) of the respondents strongly agreed 
or agreed that the evaluation efforts in their state needed to be 
revised to meet the needs of LEP students. Over 22% of the 
respondents indicated neutrality to the same question, while one- 
thind disagreed for the need of revisions to their evaluatiqn efforts. 

In order to determine whether the respondents felt their own 
state's student and employer follow-up studies of LEP students needed 
to be improved, respondents were asked to indicate positively if they 
did feel a need for improvement or negatively if they did not perceive 
a" need for improvement. . If the answer was YES, respondents 
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TABLE 1 

Respondents Perception Concerning the Effectiveness of Their Own 

State's Evaluation Process 



Question Response Opinion, Frequency '% 



*2 Your state is effective 
in meeting the /require- 
ments set forth by 
Section 112 (Title II 
Education Amendments of 
1976) in evaluating all 
vocational education 
programs • 

*3 Your state is, effective 
in eva1ua1;ing services 
and programs aimed at 
LEP students • 



*4 Your state is effective 
in utilizing evaluation 
results in improvement 
of services and programs 
aimed at LEP students. 



*5 The dvaluat4on efforts 
in your state need to 
be revised to meet the 
needs of LEP students. 



. ; L » 

* Numbers correspond to question numbers on puestionnaire on Evaluation 
Practices, 




strongly Agree 


12 


26.6 


Agree 


26 


57.7 


Neutral 


3 


6.6 


"Disagree 


1 


2.2 


Strongly Disagree 


0 


O.O 


No Response . 


3 


6.6 



Strongly Agree 


3 


6.6 


Agree 


. 18 


40.0 


Neutral 


17 


37.7 


Disagree 


4 


8.8 


Strongly Disagree 


0 


■ 0,0 


No Response 


3 


6.6 


Strongly Agree 


1 


2.2 


Agre« 


14 


31.1 


Neutral 


20 


44.4 


Disagree 


7 


15.5 


Str.ongly Disagree 


0 


0.0 


No Response 


3 


6.6 


Strongly Agree 


4 


8.8 


Agree 


13 


28.8 


Neutral 


10' 


22.2 


Disagree 


15 


33.3 


Strongly Disagree 


0 


0.0 


No Response 


3 


6.6 



were asked to provide a brief description of the improvements they 

felt necessary. Table two presents the results to this question. 

Respondents did provide some explanations for their response: 

* 

They (LEP students) would be a part of the overall excellent 
f9llow-up system we have in place, since they would be en- 
rolled in an instructional vocational education program (State 
personnel in* vocational education). 

VEDS follow-up should key the LEP student/employer responses 
separately and develop a separate report for these students 
only (State vocational consultant). 

•One, three, and five year follow-up procedures to determine, 
type of employment, salary levels and present status of 
feelings toward prior vocational servtces (State personnel in 
vocational education). 

Since there are not follow-up studies at this time, anything 
would be an improvement (State consultant in vocational edu- 
cation). 

An evaluation designed specifically for LEP students. .Because 
the number of LEP students is- small, 500+ statewide, not n|jjch 
attention ,was given in the past to evaluation. However, more 
emphasis should be given as is to disadvantaged and handicapped^- 
students evaluation (State personnel in vocational educati^^^K 

We would need to know how the business community relates to 
people (LEP studerjts) that hav^ been trained in vocational 
and related to specific job task English, Socially our stu- 
dents going through vocational* Engl ish as a second language 
still lack the social skills ^d ESL in social situations. 
This relates more to culture shock and cultural awareness 
(State consultant in vocational, education) . 

We- need to get a higher return of student follow-up studies. 
The /follow-up survey is being revised to help meet this need 
(State Director of vocational education). 

Respondents were asked in item number eig'lrt In the questionnaire, 
to send material or literature that described their state's evalu- 
ation process or thaV supplemented their answers in the questionnaire. 
Ten states, 22.22% did send materials describing their evaluation 
process or vocational programs. Eight ^of the states sending materials 




TABLE 2 

Respondents Perception Concerning the Need 
for Improvement of Student and Employer 
Follow-up Studies of LEP Studef^s 



Question ^ Percent of Total Response 



Yes No No Re s ponse 

No. % No~ % UoT* %^ 

*6 Do you feel your state's 
follow-up studies of LEP 
students need to be 

improved? . 20 44.4 19 42.2 6 13.3* 

'*7 Do you feel your state's' 
employer follow-up studies 
of LEP students need to 

be improved? 17 37.7 22 48.8 6 13. 3 . » 



* Numbers correspond to question numbers on Questionnaire on 
Evaluation Practices 



appeared to utilize both self-study and on-site procedures for the 
evaluation of vocational education programs. The self-study pro- 
cedures were used as a preparatory state for the on-site team visit. 
Data collected in the described procedj(ires rarely surpassed the 
biasic informatioit needed for identifying LEP students for reim- 
bursement purposes or for meeting data requirements of the VEDS 
forms. 

CRITICAL VIEWS AND CONCERNS 

Informal and formal interviews were conducted with a variety 
of individuals. Persons interviewed were staff at. the DAVTE, mem- 
bers of the State Advisory Council for Vocational Education (SACVE)-, 
local administrators, local instructors, locaT^idance and 
counseling personnel, students and community members. The 
description and number breakdown of the persons interviewed is pre- 
sented in Table 3. 

The total number of interview held with different individuals 
totaled forty-seven (47). Some persons were interviewed more than 
once for the purpose of amplification, clarification, or verification 
of themes and issues^ The interviews were phenomenological in nature 
^described in Chapter Three. The analysis was an ongoing process, 
The data were analyzed immediately, and inferences drawn, new 
questions raised, ancj themes [issues) were identif ied-which a'd- 
justed the focus and intensity of future interviews and obser- 
vations. 

iThe presentatior^ of data will be divided tnto the four main - 

themes. These four themes were identified by the interviewees as 

being major issues or concerns relavent to the evaluation of voca- 
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TABLE 3 

Description and Number of Persons Interv^lewed 



Description - Number of Persons' 

• * f 



Members of State Advi^nrv Pftunrn fnr 
Vocational Education QSACVE) 


2 




Staff at the Deoartment of Adul t vnra- 
tlofial and Technical Education (DAVTE) 


4 




Team leaders 


6 




Team Members 


6 




Admihistrators 


6 




Instructors 


7 




Guidance and Counseling Personnel 


5 




*»LEP Students (enrolled In vocational 
training) 


6 




Community Members 


5 
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tional education [programs serving LEP students. It should be noted 
that the interviewer was not attempting to generate standardize 
stimuli, such as test items, or questionnaire iterfis. The principal 
stimuli were considered to be those issues or concerns which were 
most relevant and natural to the interviewees. The four main themes 
emerging from the interviews are as follows: 

A. Strength and weaknesses of on-site visitation. 

B. Changes occurring in the areas of programs and services 
aimed at LEP students as a result of the on-site visit- 
ation. 

C. Expertise and composition of on-site team members. 

D. Possible changes of the on-site evaluation process 
with respect ot evaluating programs and services 
aimed at LEP students. 

Strengths and Weaknesses of the Qn-Site Visitation 

The interview data concerning the overall strengths of the on- 
y site visitation were positive in many aspects. Issues which were 
identified as strengths of the on-site visitation werer the posi- 
tive impact on LEA'S, beneficial and timely final reports, oppor- 
tunity by LEA personnel to express opinions, and the comprehensive- 
ness of the evaluation process. The Issues related to the weaknesses 
of the on-site visitation were the following: the composition of 
team members, likelihood of friends evaluating friends, lack of 
LEA preparation prior to on-site visitation, lack of team member 
preparation prior to' on-site visitation, and inability to eva^luate 
the quality of instruction and programs via the on-site process, 
Th^ following quotations tire presented as illustrative of the • 
aforementioned issues; y 

» 67 m 



77 



1 



The three phase strength; is the kind of in-^servtce effect 
*(it has) on the team members. Team members grow so much, 
just by being adle to critically look at someone else^s 
program. Also the idea that I get to take back to iny 
school will help me out The one thina^I think IHJ 

be able to use are the industrial ed, comMtencies they 
have developed here, I We got a copy the competencies) 
the department head give me (team member 

My school was evaluated last year. I gue^s the strength 
that I saw was that I was interviewed at all. I'm a 
part-time teache^r at tny college, and I was interviewed 
after my 7 p.m. class. The team member must have been on 
campus till 10:30 that ntght, Thats why I volunteered .to ^ 
go back tonight, to Interview the part-time faculty, 'Its 
important to Interview everyone (team member). 

Not only do team member get alot out of this (on-site, 
visitation) but the school does too. The report is 
easy to read, atid it's given to faculty. Faculty get to 
see the total picture, which they weren^t wtlling to do 
alot of the times. The report covers alot, and thats 
good (LEA administrator}. 

T|ie system has really macte vocational education grow In 
Illinois. Ten years ago,\vocational education programs 
were scarce. The on-site 'got LEA's off dead center. ^ 
There has been^alot of growth due to the Three Phase 
The growth isnf't only in the number of vocational edu- 
cation programs. It's good quality training that kids are 
getting (team leader). . v^;^. / 

The system is one of the best in the states. Its been 
able to make vocational education grow. It's' (The three 
Phase System) given the job of evali/atlon to v-6cational 
educators themselves. Peer evaluation's been effective 
(DAVTE staff). 

A weakness of the on-site Is the over dependence on teajn 
leaders. Team leaders are given too much flexibility. 
The team leader should make certain all the areas are ' 
addressed, but some don't (local instructor). 

The weakness I see, is the prepa»tation the school does • 
prior to the on-site; some Just try to snowball you and 
others just aren't ready. The team needs more information 
that often isn't there. I'min favor of the self-study 
that is being talked about (team leader), 
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The problem with the' system, is that it can be incestuous. 
By that I mean, it's highly likely for friends to be 
evaluating frielids. 'We have had two on-sites, emd each 
' time Tve had personal friends on the team.. That can 
be a little bias, I mean, it'^-all been very professional, 
but it just shouldn't happen (local administrator), 

• ' The ability for the team to look at the quality of the 
. programs is my- concern. I've seen the system at work, 
but we aren't looking at the real quality (DAVTE staff). 

Changes Occurring in 1>he Areas of Programs and Services Aimed 
at LEP Students as a Result of the On-Site Visitation 

The data concerning changes that had occurred in LEP programs* 

and services as a result of the on-site visitation were not varied 

The data disclosed a paucity of changes as a result .of an on-site 

visitaffon. The following statements are examples/of the issues 

related to this major theme. ^ 1 

Don't expect too much of the system, since it*wasn^t designed 
to look at these types of programs (DAVTE staff). 

The program that we have here is not a result of the evalu- 
ation we had five years ago. Five years ago if I can retail 
right, there wasn't any emphasis" on LEP students, it was 
the disadvantage and handicapped students who had the at- 
tention. The bilingual program is the result of community 
persons demanding— no asking that we provide some services 
(local administrator). 

Our school was evaluated last yfear, and I don't think pur 
Hispanic (ESL Program) was ever mentioned in the report. 
-Its doing a gj^at job, but it wasn't looked at (team , 
^ member). 

Expertise and Compbsition of On-Site Team Members 

The interview data regarding the effects that ''evaluation team 

ft ^ ' 

members expertise has on the evaluation of programs and services 

<^ 

aimed at LEP students were varied. The opinions ranged from^the 
effects bqing unsubstantial, to being critical in evaluating these 
types of programs and services. The subsequent quotes portray 
the variability. 
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When it conies to services for special needs students, the more 
administrators, and general is ts the better handle you'll have 
on their (special needs students) needs (team leader), 

I can't see how you can evaluate programs for LEP students 
if you don't have experts in that field. Most vocational 
instructors simply haven't had experience or even training for 
these kids. How can I say college is doing a good job serving 
the Indochinese or Latinos if I don't know .what can'or ought 
to be done for them. I know DAVTE doesn't know what to do 
(vocational instructor). 

I've been on an on-site anckl think the team did a good job. 
The team leader had us; she was good, interview persons out of 
our field. I'm a counselor, and I interviewed nurses, welders 
and administrator^. The mixture is good. The team efforts 
is what makes it work (local counselor). 

The team leader is the key to the evaluation. If he sees an 
area isn't being covered, he has to make sure everyone is inter- 
viewed and I guess the right questions are being asked. If 
I can see that, let's say LEP students aren't being addressed 
by Tuesday evening, then I make sure someone will cover it 
on the next day (team leader). 

The team, I believe makes a difference. I suppose Ideally we 
need someone with special needs interest on every team but 
it isn't feasible (State Advisory Council for Vocational 
Education, Member).' 

The system is overloaded already. Making sure each team has 
certain experts^ is just not possible. The team approach has 
worked (DAVTE staffj. 

I don't really _think we need to make sure every team is balanced 
The team leaders are^the check and balance (local administrator) 

Possible Changes of the On-Site Evaluation Process with Respect 
to Evaluating Programs and Services Aimed at LEP students 

The interview data related to possible changes of the on-site 

evaluation process with respect ot evaluating programs and services 

aimed/ at^ LEP studejits were diverse. Most persons interviewed fo- 

cusedh on three issues>,nthose being; the composition of the team, 

the intervi^v task in the Team Member Handbook , and role of DAVTE 

in assisting bEA's implement recommendations from the final evalu- 

uati»on report. . The fallowing quotes will further illustrate. • 
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The DAVTE doesn^t follow-up on the">%port. If they (the DAVTE) 
would follow-up and show schools how to start service for 
LEP's or any other area, schools would be more responsive. 
It*s a problem of what comes first, the chicken. or the egq? 
Springfield (the DAVTE) needs^ take the lead. The (LEP) 
studehts are out there and schools won't start unless some- 0 
one comes in. and starts asking "What are you doing in this 
area?", and if the answer ts zero. Then the follow-up has to 
be there (LEA administrator). y 

If DAVTE, knows which schools are claiming reimbursement for 
LEP students,^ then it should make certain someone on the 
team can address whatever Is being done at the school (team 
leader). 

The state already* has alot of data on LEP students. We all 
have to fill out the VEDS report. They (the DAVTE) need to 
take a look at that, the data is not being used (LEA 
administrator); 

The handbook CTeam Member Handbook) doesn^t tell members what 
to look for. It assumes team members will know how to"" / 
evaluate programs for the special needs students. You can't 
assume all members will know how to determine impact of 
services (team leader), 

EXTENT AND NATURE OF THE IMPACT 

Several data gathering techniques were used tci collect data 
reg^t^ding the extent and nature of the TPS impact on the local 
education agencies that were evaluated in FY 1980, The data were 
collected via a questionnaire, and formal and informal Interviews, 
Questionnaire on thevEffects of the Three Phase System 

A total number of 33 lEA's received the questionnaire. The 
questionnaire was designed to provide empirical data of .the effect 
tfie on-site visitation had on programs and services provided to LEP , 
students, A total number ^of 29 {87%\ LEA's returned the'questionhaire. 
However, eight of these 29 LEA's did not complete the questionnaire. 
These eight LEA*s chose not to complete the questionnaire on the 
basis that their LEA's did not have LEp studenj5<^ ^ typical respl)nse 

for not completing the questionnaire was; 
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most frequent positions were .Director Vocational Education (28.5%) 



I am afraid this is not very useful to yOu. We have so'' 
few LEP students that most of the questions don't apply' 
and, of course, the evaluation team. really had nothing 
much' to investigate as far as LEP program is con^rned 
(LEA Dire'ctor of Vocational Education). *' 

<^ 

The completed questionnaires totaled 21 (63.6%) of the total 
33)." These 21 questionnaires were used for the data analysis. 
The respondents were asked in°the first nine items in the question- 
naire to provide general back^ground^haracterisric-s regarding their 
particular LEA and staff. The first questioYinaireN^tem which asked 
for the posftion title of the respondents indicated that^the two 

\ 

and ^Coordinator of Vocational Education).' 

^Respondents indicated whether' they had ever served as a member 

of a DAVTE on-site evaluation team. Fifteen (71,2%) indicated "yes' 

# 

whjle six said they had not. Those that responded positively^ were 

also asked to indicate Jiow' many teams they had been on. Thq total 

number of teams 'was- 47, with the mean being *3. 13 teams. 

The respondents were aTso asked to^lndicate whether they spoke 

another language other than Ehgjish, Nineteen indicated that they 

did not, while only two. ind^jjated they were fluent in a language . 

other than English. These two respondents further identified the 

language antJ noted their spea?:ing ability. One' respondent rated 

his speaking ability of Spanis;h as^ "Good" while the'' second re- 

spondent rated his speaking j&bility of Slavic as "Fair". 

'* ^ 
Respondents were asked to rate the vocational education program 

in ,the"i.r districts. Over 76% rated the program in their district 

as "Above" or "Wei T above" average.. The respondent Vated the co- 

operation between vocational faculty teaching LEP* students » ^ 
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with both ,° the guidance/counseling personnel and the Vocational 

English as a Second Language (VESL).or English a§ a Second 

Language (ESL) faculty. The cooperatipn with the gui'dance/ ^ 

counseling personnel was rated "as "High" by 52.3% and was rated 

as. "Average" by 47.6% with 'the VESL/ESL faculty. ' ' \ 

Respondents .indicated the, gulfural sensitivity to LEP stu- 

cients of vocational instructors, guidance/counseling staff, VESL/ 

ESL faculty, ^nd administj^ajtors. Cultural sensitivity was defined 

as^ follows: "Ttie ability to acknowledge and appreciate the reality 

*of differences within and among ethnically or culturally diverse 

^students/' , The results indicated "Average" cu-ltural sensitivity 

of 61.9% for-administrators; .57,1% for vocational instructors and 

guidance/counseling staff, and. 38% for VESL/ESL instructors. 

. Respondents were^also asted (to the best of their knowledge) 

to identify staff pmbers which were able to conmunicate at least 

minimally with LEP students in their native language. The responses 

indicated that the following staff could communicate at least 

minimally: eight (6,9%) of 115' administrators, 'l5 (3.9%) of 377 

vocational instructors, ^5 (69.4%)^of 36 VESL/ESEMnstructors , and 

13* (13.6%) of 95 guidance/counseling staff. 

Respondents .indicated how often staff recently (within the last 

two years) had attended irT^service presentations or workshops con- 

earning issues an^ concerns of LEP students in 'vocational education 

program$^and services. Over 76% of the staff had little or no in- 

service at all, , • ; , _ • 

• " * ' \ ^ ' * . . ^ 

The remaining data in the xjuestionnai^e focused on the on-site 

visitation conducted in the respondents district in school year • 



\ 

1979-80, The first item focused on the on-site centered on the 
on-site visitation exit interview. Data was gathered concerning 
what persons involved with LEP' students were invited ^to the exit 
interview and also how many attended. Administrators were the 
group most 1?^equfently invited 66.66%, followed by the vocational 
faculty and board members both at 42.85%. Table 4 further 
summarizes the data relative to the attendance of staff at the exit 
interview. 

Respondent identification of those persons involved with LEP 
students that received the final report of the on-site evaluation 
are summarized -in Table 5. It should be noted that vocational in- 
structors were identified as receiving the report in 76.1% of the 
LEA'S while administrators, guidance/counseling staff, and board 
members had received the report ip 15 of the 21 responding LEAs 
(71.4^). , \ J 

1 ^ Re spondents identification of the extent to which persons in- 
volved with LEP students were familiar with the information in- 
cluded in the evaluation report concerning LEP student programs 
and services are' summarized in Table 6. Administrators were rated' 
7.42% as being "Moderately" to "Extensively" familiar with the 

evaluation report. The second most familiar ("Moder'ately" to 
"Extensively") were the guidance/counseling staff at 61.90%. , 

Respondents were asked to Vate their agreement with the 
following statement: "I think the DAVTE on-site evaluation helped 
us to improve our entire 'vocational education program" All 21 
(100%) respondents "Strongly agreed" or "Agreed" with the statement. 
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TABLE 4 

IDENTIFICATION OF THOSE PERSONS INVOLVED WITH 
^ LEP STUDENTS THAT WERE INVITED TO THE ON-SITE 
^ EVALUATION EXIT INTERVIEW (N=21 LEAs) 



Group 



Invited to Exit 
Interview 



Number That 
Attended Interview 



YES 



NO 



NO RESPONSE 





Freq'. 


1 


Freq. 


% 


Freq. 


% 
\ 

9 5 


# Attending 


Administrators 


14 


66.6 


5 


23.8 


2 


34' 


Vocational ^ 
In$,tnjctor^ 


9 


42.8 


10 


47.6 


1 




15 


/eSL/ESL Staff 


^ 


14.2 ■ 


^4 


66.6 


' 4 


19.0 


' .1 


'fiuidancje/ . * 
Counseling Staff 


8 


38.0 


' 11 


52.3 


2 


^.5 


11 


Par^ts 


2 


9.5 


15 


71.4- 


4 


19.0 


2 


Board Members 


9 


42.8 


9 


.42.8 


3 


14.2 


1^ 


Others (Advisory 
Council Members) 


i 


4.7 


13 e 


■ 61.9 


7 

♦ 


33.3 


2 ' 
















t 










* 
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^ TABLE 5 

IDENTIFICATION OF THOSE PERSONS INVOLVED WITH 
LEP STUDENTS WHO RECEIVED THE FINAL REPORT 
OF THE ON-SITE 'EVAlBat I ON (N=21 LEAs) 



Group 



i 



Received Report 





YES 
Freq. 


% 


NO 
Freq. 


% 


NO 
Freq. 


RESPONSE. 
% 


Administrators 


15 


71.4 


4 


19.0 


2 


. 9". 5 


Vocational Instructors 


16 ' 


■76.1 


3 


14.2 


2 


9.5 


VESL/ESL Staff 


7 


33.3 


10 


47.6 


4 


19^.0 


Guidance/ 
Counseling Staff 


15. 


71.4 


5 • 


23.8 




4.7 


Parents 


1 


4.7 


4 


19.0 


2 


. 9.5 


Board Members 


15 




14' 


66.6 - 


6 


28.5 


Others (Advisory 
Council Members) 




4.7 . 


11 ■ 


52.3 


9- 


4T8^^ 
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, TABLE. 6 

V 

IDENTIFICATION OF THE EXTENT TO. WHICH PERSONS INVOLVED WITH LEP STUDENTS WERE 

] 

FAMILIAR WITH THE INFORMATION INCLUDED IN THE EVALUATION REPORT CONCERNING 

» 

LEP STUDENT PROGRAMS AND SERVICES (N=21 LEAs) 



Group 



Extent of Familiarity 



• » 


Extensively 
Freq. % 


. <r ^ 

Moderately 
Freq. % 


Little 
Freq. 




None 
Freq, 


% 


No Response 
Freq. % 


Administrators 


7 


33.3 


'8 


38.0 


3 


•14.2 


1 


4.7 


. 2 


. 9^5' 


Vocational Faculty 
VESL/ESL Faculty 


2 
4 


9.5 
19.0 


8 

6' 


38.0 
28.5 


8 
5 


38.0 
23.8 


;.) 


4.7 
14.2 


2 
3 


9.5 
14.2 


Guidance/CoOnsel ing Staff 


4 


19.0 




42.8 


' 4 


19.0 


2 


9.5 


2 


9.5 


Board Members 

/ 

Par^ts 


•2 


9.5 


9 


42.8 


7 


33.3 


1 


4.7 


2 


' 9.5 


1 . 




1 ■ 


4.7. 


6 


2.8.5 


6^ 


28.5 


7 


33.3 


Others 

Not Identified ' 


0 . 


•0.0 




0.6 


. 1 


■ 4.7 


4 


19.0 

> 


15 - 


'71.4 


Advisory Council 
Members 


> 

0. 


0.0 


1 


0.0 


0 


0.0 


• 0 ^ 


0.0 
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Specifically, 7 {33.3%) responded with '^Strongly agree" ^nd 14 
(66.6%) responded wijth "Agree". 

In addition to the preceding questTopnaire item, respondents 
were also asked to rate thefr agreenient with the following statement: 
"During the DAVTE On-site evaluation our UEA go^ some good 1'deas on 
how to improve the vocational education programs and services aimed 
at LEP students." Nearly 62% agreed or strongly agreed while 33.3% 
disagreed or strongly disagreed with the statement. The data are 
further suimarized in Table 7, The succeeding questionnaire item 
was also rated for agreement; "To the best of my know^ledge the DAVTE 
evaluation^eam members^who interviewed, our LEA were knowledgable 
in the area of LEP students." Over 52% agreed or strongly agreed 
while 28.57% disagreed and 19.04 indicated a "Don't knOw" response. 
This data is summarized in Table 8. ' , 

Respondents were^lso asked a question regarding an increased 
awareness of program improvement resurces. The following statement 
was posed: "Through the on-site evalue(tion, our LEA became aware of 
resources and services that will/are being used to improve programs 
and services for LEP students." Oyer 38% "agreed" or "strongly- 
agreed"' while 52.1*% "disagreed" or "strongly disagreeq^.V Respondents 
were also asked tO what-jextent have these resources- been utilized? 
Respondents selected the response options as follows: "moderately" 
28.5%, "little" 23.8% and "not a-.tall" 28.5%. The data is'summarized- 
in Tefble 9. . ' ^ . ' ^ 



TABLf 7 

RATINGS THAT DURING THE DAVTE'ON-SITE 
EVALUATION THE RESPONDENT'S LEA GOT SOME 
jPEAS'-ON HOW TO IMPROVE THE VOCATIONAL EDUCATION 
PROGRAMS AND SERVICES AIMED AT^EP STUDENTS (N=21 LEAs) 



Question 



Response Option Frequency Percentage 



14. 



During the DAVTE on- 
site evaluation our * 
LEA got some good 
ideas on how to 
.improve the vocational 
education programs 
and services aimed 
af'LEP students. 



Strongly Agree 


2. 


9.52 


Agree 

it 


11 


, 52.38 


Disagree 


6 


28.57 


Strongly Disagree 


1 


4.76 


Don ' t Know • 


1 • 


4.76 



* Number corresponds to questipn number on Questionnaire on the Effects of 
the' Three Phase System, Appendix B ' - 
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TABLE 8 

RATINGS OF tuE DAVTE EVALUATION 
TEAM MEMBER'S -KNOWLEDGE OF LEP PROGRAMS V 
^ AND STUDENTS (N=21 LEAs) 



Question 



Response Option Frequency Percentage 



★ 

15. 



To the best of my 
knowledge the DAVTE 
evaluation team Members 
who interviewed our LEA 
were knowledgabl^ in the 
area of LEP students. 



Strongly Agree 2 

Agree 9 

Disagree 6 

Strongly Disagree 0 

Don't Know 4 



r 



9.52 
42.85 
28.57 
0.0 
19.04 



* Number corresponds to question number on Questionnaire on the Effects of 
the Three Phase System, Appendix B 

V • , 



Respondents were asked if thelr.distn'ct had requested DAVTE 
consultant services to assist with implementing the recommendations 
regarding LEP siudents. Fifteen (71.4%) of the respondents 



selected response option "YES" while 4 (19.0%) ^rrdicated "NO". If 

respondents answered "YES", they were asked to describe the services 

r^eceived. Four descriptions were provided: 

Consultant to' investigate the lack of students in our 
home economics program. 

Special needs consultants. ^ « 

Staff consultant gave vtorkshop for the Staff, studying LEP 
student programs. 

Consultant services - Industrial Education 

Respondents were asked to rate their agreement with the following 
statement: "In your opinion, was adequate time spent by the evaluation 
team in examining the programs and services related to LEP students?" 
Over 52% "Agreed" or "Strongly agreed" whila 23.81 "Disagreed" or 
"Stroflgly disagreed" with the statement. Five (23.8%) indicated 
that they did not know. Respondents wl^e also asked to rate their 
agreement with the following statenrent:! Many of the changes that 
will /are being, made in our vocatiwial edj^ation program and services, 
for LEP students. have been the result of iVpormation provided by the 
Von-site evaluation report." Thirty-thrl^(^ percent "Agreed" or ^ 
"Strongly agreed" while 6-6.6 per cent "Disagreed" or "Strongly disagreed" 
with the statement, The preceeding data are further illustrated* 
in Table 10. 

Data Gathered Through Interviews 

Following the Quest1onnaire-on the Effects of the^TPS, inter- 
views were conducted w.ith personnel of three selected :LEAs, team « 
leaders, LEP students and regional, vocational administrators. The 

• '81 ^ . 



— TABLE 9 
RESPONDENTS RATINGS ON THE AWARENESS 
AND USE OF PROGRAM IMPROVEMENT RESOURCES 



^ AND SERVICES (N=21 LEAs") ■ 
X • "" 






Question 


Response Option 


Frequency 


^ / 

Percentage 


16. Through the on-site 

cVa 1 Ua t<4 Oil , our Lun 

becajne awa^N^pf ./ 
resources and services] 
' that> will/are being ^ 
* used to improve programs 

and services for LEP-* 
^students. 


— y- 

Strongly Agree 
Agree 
Disagree 
- Strongly-Oisagree 
Don't Know 


1 
7 
6 
6 
1 


—• :T- 

4'. 76 

33.33 
28.57 

• 4.76 


To what extent have 
these resources 
been utilized? 


Extensively 

Moderatly 

Little 


0 
5 


0.0 

28.57 
23.80 




None at All 


6 


28.-57 


9 ^« 


No Response 


4 • 


,19.04 . 



* Number corresponds to Jiuestion^mber on Questionnaire on the Effe,c^ of 



the Three Ph^se System, Appeyidix B 

/ 
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• TABLE 10 

f. RATINGS ON THE TIME SPENT BY THE EVALUATION TEAM 
OM LEP PROGRAMS AND SERVJCES AND ON CHANGES RCSULTING 
, ' FROM Ijilf^ORMATION PROVIDED BY THE^OTl-SITE EVALUATION REPORT (N=21 LEAs) 



Question 



Response Option Frequency Percentage 



.18. Jn ydur opinion, was adequate Strongly Agree 
time spent by the evaluation 

team in examining the programs Agree 
and services-related to^iEP 

students? ^ Disagree ^ 

^ J * / Strongly Disagree 

^ ' Don't Know 



19. 



Many of the changes wil >/are 
being made in bur vocatiorhjl 
education program and- 
services for LEP students 
have been the result of ' 
information provided by ti 
on-site evaluation i/fepprtl 



.Strongfly Agree, 

)\gree 
' Disagree 

Strongly Disagree 
'DorT^'Kriow 



1 

10 
3 
2 
5 

2 
5 
8 
6 
0 



4.7 
47.6 
14.2 

9.5 
23.8 

.9.5 
23.8 
38.0 
28.5 
0.0 



** Numbers correspond to question numbers on Questionnaire on the Effects of 
the Three' Phase Systeiii, Appendix B • * *. . 
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purpose of the interviews were to acquire a further in-depth 
spective of the impact of the on-site visitation had upon programs 
and services provided to LEP students. Based on the responses to 
the questionnaire, and a review of the LEA*s on-site evaluation re- 
port, interviews were condtlcted with an emphasis to clarify and 
amplify data gained through the questionnaire and the evaluati9n 
report. 

Three types of LEA*S were ^eleSted for further study, one 

suburban secondary school' district, one suburban »area vocational 

• « — 

center and one urban community college. Selection was based on 

the criteria discussed in Chapter Three, 

The Suburban Secondary School District 

The suburban secondary district consists of 1:wo schools, one * 

established in 1926, and the Qther in 1959. Thisschool district' 

inclucles 16^^ square miles of industrial community wrapping around 

a iji,ajor airport serving the largest city in the state.' The school 

district's population is multi-ethnic including students of German, 

English, Scotch, Irish, Italian,* Poltsh and Scandinavian anceSLtry, 

f 

iKTitf^jhe lea's One and Five Year Plan fot* vocational edu- . 
edition, there is a growjng minority population of.Spanish-speaJ<ing, 
students- and two and one-half ^percenV of the families ar.e below^ 
'poverty *tevel . Tfje soeio-q^Onomic level of the students may be 
' ^corisid§if*ecit1ower-middl& class, ' • > ^ . ' \ ^ 

' ^ ^ '|te,jrchboV'^diS'tj?4ct offered VQcationaT prpgrams in.*he^fol- */; 

lowing areas? , ^ome pGOjripmicsi, business, mctPl^iYig[ and nianagenfent*J 
. a>id tndtfetrial. 




The- on-site- evaluatj on repor^^d, seated tk$t\ :M . 



the mandates of T.itle IX ^had been reviewed and implemented through- 
out the vocational education programs. However, in-service activities 
related to eliminating sex bias/roVe stereotyping' should be offered. 
\ The report al^o stated that the mandates relative tx^disad- 



J. 



vantaged and handicapped students had been -reviewed and implemented 
'throughout the vocational education program. However, vocational » 
faculty should be involved in the preparatfon of Individualized . 
Education -Programs (lEP's) 'for special education students main- , 
streamed in their program (P.L. 94-14?)., Reference to LEP stuclent/^ 
issues;' concer*ns, programs dt services were not mentioned ^in, tbe final 
on-site evaluation^repor.t. No conclusions were writteh Voncerning . , % 
identificatjpn criteria or special services to LEP students.^ 

This'data was supported, by the responses in the Questionnaire 
on .the Effects of the TPS. the questionnaire responses tor this ^ . * • 
particular LEA indicated^ that this LEA indeed, Tiad no^t received aijy^- ^ . 

■f- • ■< ' '.^^ '."'^ 

ideas Qn how to improve programs and serv.ices tOxLEP students. C v' 

In , addition, the re^ponj^es^ajso indicated that the-lEA.had npt i ' I 

. bfeC0m6 aware ny^l^ny^am ■;mniAAw/an,^^«4. v^«^rM.v.^«^ *^U^/^..«l^ 4.U^ ^« 4 4*^ «^ 



/are oY prDgram ii^prav^raent ^resources ^{hrougft. tbe Qn-si*^"^ 
NffnaV^valuatipn-repOrt. ^k-^* ^ ' ' : > ^ 



o ^ ^ ' Oatdyg^thered^throifgh^th^./hten^fews sulipo^ data gathered,- - 

' ' ' t * ^ /: • 1^ - ■ ' ' • " ; ^ 

\ V ^^'^'thr^g]f^.eh^^ analy^i^^^ tfte fipal .on-sit^ eva.lu-'^ - 

ation reports. The vocational director stated 'that he was "ne^er . 



f ^ % ""^asked. any specific questions dea'lfng with LEP studeatg*." the 
* ' principal and the vocational director both agreed that the streng^ 
of the entip TPS was. the mixture of teanf members. However, it 
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was #1 so pointed out by the vocational director, that there was a 
potential problem in "knowing who is coming to evaluate'the >pro- 
gram", The'objecttvity cff team members ^valu^ting professional 

' peers or friends according to the vocational director, 'coutd be 
assured if the Team Leader kept the team members on t?isk, 
' Changes had occurred within the last two years^in the gtrea of 
programs and services aimed at LEP 'students. The major chang? * 
was an increase of ESL faculty working with vocational faculty in 

^modifying, and integrating instructional, lessons for LEP students. 
The district as a whole had formalized a stronger commitment toward 
ESL practices dueNto highly motivated and energetic ESL faculty. 

The team composition fof this particular evaluation was a 
"well balanced team"- according to the Team Leader. This was * 
supported'by the vocational director:' "The team I believe was 
knowledgeable in most areas. The team leader Was an excellent person 
she did a good job|keeping the team on task". 

The recommendations provided by several administrators at th*is - 



school were 'the following; \^ 

I suppose first of all, there must be an agreement that ' 
there is a problem, people- in Springfield- need to make 
' Team Leaders aware of the problems of LEP students, Team 
Leaders have a responsibility along with the Team Memter^ 
Handbook to ask questions. There ts a danger, not alf ' 
schools need to do the same th-ing for LEP*s,,, but hsk 
questiohs when it's applicable. R)r instance, if'a 
district ts claiming LEP students, then the team needs 
to ask .some specific questions (LEA administrator). 

• The one' recommen<iation is to make certain* that some 
generalists aj^e on- the team, Administraton^ and voca- • 
tional education generalists possibly have a broader 
perspective. They deal with larger issues on a regular 
basis (LEA adminis'trator), 



The Team Leader when interviewed suggested the" following 
recommendation: * / - ' . ^ 

The improvement will have to depend on the Team Leaders, " " 
• The have to make certain all areas are covered in an evaluation. 
It's hard to get a team which wtll be experts in all fields. 
It's up to the Team Leader (Team' t'eader) . 

The data gathered concerning the effect on the on-site evaluation 
had upon the secondary school indicated minimal impact. School person- 
nel perceived recent changes within their district as receiving impetus 
from the districts ESL faculty. Impact on the^entire district's voca- 
cational education programs was seen as positive and extensive. 



The Surburban Area Vocational Center 

The Surburban Area Vocational Center (AVC) provides voca- 
tional opportunities primarily, for the juniors and seniors of the 21 
participating high schools. Enrollment is on an ejective basis from 
each local high .school. There are approximately 900 student learning 
stations available during a single shift per day. The'llth and 12th 
grade enrollments in the 17 participating high schools is approximately 
'12,000 students. The AVC fs organized under five major areas yvith a 
total offering of *19 specific vocatfonal clusters. The five major 

ft 

areas are:_ . ' ' 

1. Applied Biological and Agricultural 

2. - Business, Marketing and Management 

3. Health 

4. • Home Economics Occupations ♦ ' 

5. Industrial Oriented j) 

The on-sit'e evaluation report stated that the student services 
personnel were to be conmended for establishing a formalized system,, 
for identifying disadvantaged and handicapped students. The report 
however, did not mention the programs or services directed at LEP 
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students. No information was given with respect on how to start 

or improve current' prattices relative, to LEP students. ^. 

• " ) 

This lack of mention of LEP students in the evaluation report 
wper' supported by the responses contained in the Questionnaire on 
t^e Effect of the TPS. The respondent disagreed thatrthe LEA had 
gained some good "ideas^ relative to LEP students, and also dis- 
agreed that the LEA had become. aware of resources or services, that 
would be used to, improve programs and^ services for LEP students • 
Further, the respondent disagreed that the changes being^made, in 
the LEA*s vocational education program an^ services for LEP stu- 
dents were the result of information provided by .the; on-site evalu 
ation report. . , 

Data gathered through the interviews reflected the data found 
in the questionfiaire and the evaluation report. The interviews 
with the director of the AVC, assistant director, vocational co- 
ordinator, and the director of the assessment center did not per- 
ceive the ^on-site evaluation as being yery hel>^ul relative to 
LEP students. Emphasis appeared to be placed 'on the usefulness 
ofj^he self-study which took place in lSr78-79 (the year prior to 
the on-site visitatibn) . .The self-study was seen 3s being very 
productive. The self -study had been d^esigned tp parallel to 
the DAVTE on-site evaluation. The same program components were 
used, tind also questions from the "DAVTE Team Member Handbook" 
were taken Verbatim, Within the self-study report for fiscal 
year .1978-79, recognition was given for the need to gather voca- 
tional ^sessment and educational materia^ls to better serve LEP 
students • 

^•88- . • 



The interviews with AVC staff reflected the following per- 



.spectives: 



The str'ength (of the on-site evaluation) lies in- the in-service 
,of the team members. Team members are able to frave oositive 
'relationships with insHructors, so they (instructors) can ex- 
press themselves. The^ on-site makes people feel like someone 
gives a damn- (LEA administrator) < 

The advisory manner of the team is good. Peers evaluating^ 
peers is the strength/ What needs improvement is that tearii 
members don't ktiow what to look -for. Dissemination is not 
there. There hasn't been a great deal of ^information- for 
modification or sample materials. With special needs stu- 
dents, information on how to m&di fx programs, materials 
that are avaola'ble, are not being' disseminated to team 
members (AVC administrator). * , 

The biggest weakness is that there isn't any. follow-up by 
DAVTE [AVC administrator). 

When it comes >to serving students'with special needs, the 
emphasis has to come from the s^ate. The on-site last year 
didn't place any on LEP students (AVC administrator). 

The impetus or the motived for 'change, when it concerns LEP 
students are several. First, there's the reimbursement from 
the state. , Second, the .incentive that we-^ee1 all kids should 
succeed. ^ You can't turn awaj kids, f or ,any reasons;. Third,, 
the laws require you to serve all kids, like Publi^'Law 94-142. 
You have the responsibility to do so and to do it well. The 
last me is the self -study. We know what we were doing, for 
all kids, but our weakest ar.^a is serving LEP students. We 
all know what we are doing, 'it's the self-a(wareness that helps 
(AVC administrator). 

There is a definite need for improvement of the on-site. The 
problem is that there is little understanding of what L£P means. 
Many people feel that LEP only refers to Hisparfics not^he man> 
Vietnamese, Laotions. What is needed are P.R. activities.^ Both . 
team members and team leaders need ta be made aware. The change 
needs to start with the state (the DAVTE). The evaluations have 
really focused on the handicapped students andon sex biasis, byA 
no one, the state hasn't placed enough (eifiphasis) bn how to or ^ 
what to do for LEP students (AVE adhiinistrdtor) .\ ^ * ^ 

The fact that no one has held back on funding for jiot reacting 
to recommendations is the biggest weakness of the system. The 
report is not taken seriously. There is noiTany seriotfs fglloW- 
up by DAVTE (Team leader). . 

'89 • •: 



\^ The data gathered concerning the effect «of -the on-.site evaluation 
on the AVC indicated little imp'^ct. AVC personnel attributed recent 
improvement^ in programs and services to LEP students-^to an internal 
self-evaluation. Effects on the overall vocational education programs 
were seen as positive but limited. ^ 
The Urban Community College 

This; urban community cgllege which is southeast of Chicago, 
serves an area of approximately 150,000 population. It serves an 
area which is multi-ethnic including students of German, English, Italian 
Polish and Scandinavian ancestory. There is also a sizable Hispanic 
community which has been a* stable force'in the conununity for several 
generations. This Hispanic populatioti has recently been noticeably 
growing according to census data. The soci o-econiomic level of 
students jn the college- may be considered lower-middle class. 

The on-site evaluation report did 'not contain s'pecific conclusions 
relative^ to programs and services aimed at LEP students. However, 
there were several statements relative to the mainstreamin^of 
handicapped and disadvantaged -students in the evaluation report. The 
report noted that faculty fe*lt that mainstreaming of ''handicapped and 

^ r- ^ 

^disadvantaged. studentSvWas no€ being accomplished and that it \vas 
not viewed as either necessai^y or beneficial to the students involved, 
TrKaddition, the report noted that several faculty members interviewed 
expressed concern relativQ^to mainstreaming. It was theiV optnicfti 
tf^jit this practice was not td be encouraged. 

^ Tfle tnformatton collected in the ques'ttonnaire reveal ed*th?it the^ , 
respondent agreed that the LEA had gained some good ideas relative 
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to LEP student programs and services, and also agreed that the LEA 
.had become awaVe of resources and services that w^re being used to 
improve programs and services for LEP students. In addition to this, 
the respondent also agreed that adequate time was spent evaluating 
LEP programs and* s.ervices and also that changes made in the LEA's 
vocational e.ducati-^ programs and services -aimed at LEP students 
had been ^that result of Information, proC'idjed by the on-site 
evaluation »re^ort. ^ * ^ - 

Th^ fact that no mention was made concerrj'ing LEP programs 
an^d ^rvices in the evaluation report but yet, the que'^tionnaire 
responses indicated that the on-site evaluation had produced positive 
results was clarified through interviews' with the community college 
personnel. The interviews revealed that indeed the on-site report 
f.or fiscal year 1980 had not effected programs arid/services provided 
•to LEP students. The conflict in data arose from the fact that the. 
last on-site revaluation (5 J^^^ Pi^ioi" to the evaluation of FY 1980) 
did, have a positiv'e effect on programs aimed at LEP students. The 
respondent perceived the programs and services provided to LEP 
students as a direct result to the on-site evaluation held in 
fiscal year 1975. ' " ' . 

" the interviews reflected the following perspectives; 
/ • ' 

The on-site five yefirs-ago- started^as^own^fre rocid in 
helping us. deal with these (LEP) students. This last year, 
the evaluation addressed if to a degree. I don't thirtk 
. it ever got into it as. far as it should have... In the 

» last tweryears there has been some growth in tfiis area. 
Especially with e.nrollment of these (LEP) students in 
technical prog>*ams. Any recent changes have been because 
of the Career Oean (Community College administrator). 
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"^he Three Phase System got us off of dead center. Five years 
ago we weren't doing anything. It got us started in thinking 
* of what has to be done with none English speaking persons 
(Coimunity College administrator). 

The strength of the evaluation is the way the data and Information- 
. is gathered. It gets at the strengths and weaknesses of the 
. local agencies. The mixture of people, and how, they are expected 
to look at a broad basis "is good. The mixture and cross-over 
in the interviewing process can get a wide* area of information... - 
the weakness is the prior preparation to the on-site evaluation. 
The team leader nee^ds to be given more information, to help 
identify strengths and weaknesses. Also the timing of interviews 
is realTy rushed. Don't increase team members, but maybe more 
time for interviews^ Perhaps, start Cising self-study, not as 
extensive as the North Central self-studies (Team Reader) , 

Every team must look at what is happening for itP stutlent^ 
Orientation must take place for all members. Team leaders 
need to be made aware to make an effort to assign team members 
to look at these (LEP) activities. The Team Member Handbook 
and also the information given to team leaders need to add some 
' direction for LEP programs, . .The DAVTE needs to revise the"^ 
Team Member Handbook^'and survey instruments (Preliminary 
Evaluation Instruments). Are limited English speaking students^ 
'being helped with every institution? This needs to be askfe4^ 
Are these students provided with a ladder to help them move on 
particularly from ESL \nd continuing edi^cation classes into 
career areas (Community ollege administrator). 

An awareness of team members by the team leader will dictate 

how much emphasis to place on different things. The team 

leader has to take responsibility to ask "What about LEP 

programs and services?" The team leader is the key , (Team leader). 

' The data gathered concerning the effect of the on-site 

evaluation on the community college indicated a si^lfi^ant effect. ^ 

Interviews with the community college personnel indicated that 

improvements ii|i programs and ser*rices to LEP students were a 

resul't of the on-Slte evaluation of FY 1975. Effects on, the overall 

vocational education program were seen as positive andextensive. 
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CHAPT€R V 

. • SUMMARY, DISCUSSION, CONCLUSIONS,. 
AND RECOMMENDATIONS-^ 

OVERVIEW • , "T ' 

The purpose of the study was to projMde a better understanding 

of *the processes used by the Illinois DAVTE in ev,^luating voca- 

tional education programs and services aimed at limited English 

proficient (lEP) students^in Illinois, This\was accomplished by 

developing and portraying a comprehensive picture of the current 

evaluation process. Included in the body of 'data were the views 

and concerns of several groups who are involved in arnl affected, by 

the on-site evaluation process fo^^ vocational education prograjns in 

plllnnois. Data, views, concerns., and improvemeot-oriented recom- 

mendations Were collected from Team Leaders, Team Members, local 

administrators, teachers, counselors, students and community members 

as wejl as4)AVTE personnel. Responses to "a series of four major 

research questions were soLjght; 

1; What procedures has the DAVTE-andertaken in order to 
*ev£rluate programs and services aimed at LEP populations? 

2. What are other states doing in evaluating services apd 
programs aimed at LEP populations? 

3. • What are the views and concerns of the DAVTE staff , \ 

lo^al administrators and board ^embers,, instructors, 
students and community members with respect to the 
evaluation of programs and services aljned .at LEP 
populations? 

^ • 

4. What is the extent and nature of the impact on local 
education agencies that were evaluated in FY 1930 ' * 
and are serving LEP students? 

t ^ ' . ^ / 

A variety of procedures were used to collect the data utilized 



in the study: 
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1. Participant-observation, document analysis and 
iaterviews were conducted while the principal 
investigator participated in three on-site evalu- 
ations of local education agencies providing pro- 
grams and services, to LEP students, 

2. A questionnaire was "clevel oped to obtain. input 

t > ' from state vocational education directors^ re- 
^ garding the current practices of states in evalu- 
^atin^ programs ^nd services aimed at LEP students. 

3. Interviews were conducted with 47 individuals to 
discover and verify the^views and concerns regarding 
Ahe present on-site evaluatibn system. The inter- 
views included: DAVTE staff,' local administrators, 
instructors, students and community members, i.e., 
employers, parents, Advisory CounciT members, etc. 

4. A questionnaire was developed to obtain input from . 
selected LEA personnel, regarding the impact on 
programs and services provided to LEP students t^at 
were evaluated by the TPS in fiscal year 1980, 
Follow-up phone and personal interviews were con- 
ducted to acquire a further in-depth perspective 

, of the impact. In addition, document analysis was 
conducted to consider the composition of the on-, 
site evaluation team, recommendations, made by the 
team, changes in the One and Five Year Plan, and also 
the numljer of LEP students being servecT by the LEA, 

DISCUSSION AND CONCLUSIONS ^ — ^ . 

.Evaluation Procedures of the niinois, DAVTE 

' ^ ^ ^ 

In 1971-72 the Illinois DAVTE developed and implemented a formal 

.planning and evaluation system to insure the'maintenartce, growth, 

and quality of al_l_ vocational education program? and services 

(Illinois State Board of Education, 1980). In order to fully under 

stand the procedures of this Three Phase System (TPS) several types 

of data were collected (See Appendix C, for a full description of 

thesystem). Data was collected through participant -observation in 

three aetueil on-site .evaluations, review of .relevant documents and 



instruments, and also through interviews with Team Members and 
^Leaders, LEA personnel, and staff from the DAVTE. 
• The data indicated that indeed, the TPS is well received by 
the LEA'S. Personnel at the local education agencies^^and^ the 
Team Men^en^ percetved the overall on-sit,e evaluation as a positive 
Snd useful activity. These positive feelings toward the overall on- 
site evaluation appear to dissipate somewhat with particular refer- 
ence to the^use^uj/ess of the on-site visitation in evaluating pro- 
•grams and servrices aimed at LEP students.. 

^ The procedures and instruments^^ provided by the on-site evaluation 
did not a[5pear to assist either the Team Leader or Members in as- 
sessing the quality of programs serving LEP students. The. most 
specific evaluative task under the Student Services Section of the 
Team Member Handbook focused on the "appropriateness of criteria 



tised to id 



sntify disadvantaged, handicapped and limited English 



proficiency students and the impact of additional* services provided 
to these students," However, little information or guidance was 
given to Team Leaders orwMembers in order to determine appropriate 
identlficaton. criteria or substjintial impact. The process seems 
to rely very heavily upon the knowledge of Team Leaders and Team 

Members in determining the extent and quality of programs and se*r- 

i 

viyCes for LEP students. 

Specific information and guidelines provided by the DAVTE xon- 
Cfirning programs and services fpr lEP students fs very .limited. 

« 

Currently, the most substantial piece of information provided by 
the DAVTE to LEAs and to Team Leaders is a handbook developed by 
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Lopez-Valadez (1979) entitled: Vocational Education for the * 
Limited English-Speaking: A Handbook for Administrators . While 
this h^'andbook provides an excellent overview and introduction to 
funding resources and initial programnatic cor\cerns, it does not 
provide indepth^information regarding theoretical, methodological 
and programmatic issues relative to providing services to hEP 
students. ' - 

Since the efforts of serving L£P students in vocational education 
is new, there is very little experience to draw on, and very little 
written on which to base a conceptual framework (Hurwitz, 1981). 
, Most of the past^professional experience of the .DAVTE staff , Team 
Leaders and Members, and LEA personnel generally appear to be very 
limited relative to programs and services for LEP students in voca- 
tional education. This void in experience in providing services* to 
LtP-^dents clearly affects the comprehensive and quality of pro- 
grams^provided, as well as how tKese programs are evaluated. 

The scope and size of the present TPS may also affect its 
ability to. do a thorough evaluation. The TPS" was f ielid-tested at 
seven sites during 1^70-71. The 1971-72 evaluations included 70 
LEAs, the 1972-73 evaluations expanded to 116 LEAs^ 72^ evaluations 
were evaluated during 1974-75, 78 were evaluated during 1976-77, <153 
were evaluated during 1977-78, 150 LEAs were evaluated in 1978-79, 
and 148 LEAs were evaluated irv 1979-80 (Illinois State Board of ^ 
Education, 1980), According to the l'978-79 Composite Report , the 
148 LEAs evaluated in FY 1979 included; 129 secondary schools, ' 
and >ost»secondary community colleges, and 10 state agencies in" 
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qorrections, mental health. and rehabilitation services. Teams * 
ranged from 1 to 33 members in size 'and involved over 750 tbtal 
Team Members. * 

The TPS has provided over the last decade beneficial evaluation 
result's which l^ave assisted in ^the overall development, planning 
and accountability tasks of the'DAVTE. However, in '^lew of the 
steady increase of LEAs that are being ^evaluated each year, and' 
also given that three-day on-site evaluations with small teams 
are the primary method of evaluating LEAs; it is the/^efore doubtful, 
/Whether the on-site evaluations can produce thorough, and compre- . 
hensive evaluations. Consideration must be given to the fact that, 
fiscal resources^ are probably insufficient for conducting "true" 
comprehensive indepth evaluations. 

Evaluation Procedures of Other State? • ' 

The questionnaire which was mailed to state vocational education 
.directors was designed to provide some empirical account of how 
many states have a separate or specific evaluation process for 
evaluating vocational education programs serving LEP students. 
More specifically, the intent of the questionnaire was to identify 
tf\e nature and ej^tent of evaluation practices in ^her states^ 
Data were obtained via questionnaire responses from 41 states andj 
four territories* Additional data were also obtained from state 
education agertcy (SEA) personnel M\i follow-up telephone interviews. 
;^ Results from the questionnaire indicated that the vast majority ^ 
of states' (91%) djd not have. a separate evaluation process for 
evaluattng. programs serving. LEP students. This data is consistent 
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with a^study undertaken by the National Institute* of Education 
(NIE). The study by NIE Reused t)n S'igntficant consequences of 
selected changed fn federal vocational education legislation 
adopted in 1976 (Education Amendments of 1976, P.,L. 94-482). Ihp 
study's interim report by NIE (1980) in speaking about how sta-tes 
were evaluating prx)grams for special populations stated the^fol- 



lowing: 



Less attention has been given to the fourth requirement, 
laid down by the regulations, evaluating the results of 
additional services to special piopulations, > the pro- . . 
gram, review. process typically e^camtnes- the attention * ' 
giveri to special populations, but focuses on access, 
-not the results of, vocational programs and services. 
- (National Institute of Education,\1980, p. V-12), x. 

H^Data from the qu^ionnaire indicated that 8555 they^ponden^s 
felt t*heir states were e/fective in meeting the evaluation* require- 
ments set forth by Section 112 (Education Amendments of 1976). Fort^.- 
six percent (46%) of the respondents indicated that their state 
was effective in evaluating programs and services aimed a\ LEP stu- 
dents. Further 33% of the respondents felt their- state was'-ef- . 
fectiWev|ig^utilizYlig evaluation results in the improvement of 
programs and services aimed at LEf students. Overall these data 
reflected an op^timistic and positive vie.w of state efforts in ;this 
area. Howler, 37% indi catted that their efforts needed to be re- 
vtsed to meet the needs of LEP students, . ; i 

Stfites tn general are^aluating and collecting. data on programs 
atid services aimed at LEP students in a limited fashion. The types' 
of data* collected address access and criteria- for identification. ^ 
The data do not rei^lect impact of vocational education programs, 

98 ■ ■ 

' y ■ .100 



Data collected is reflective of the information needs required by 
♦ 

the Vocational Education Data System (VEDS). The VEDS was a result 
of JheJ976 Education Amendments (Sec. 112). The evaluation require- 
ments (Sec. 112) intended to promote a rational planning and-systemat 
national evaluation process. * Information in the VEDS inclu'des: 
instructional expenditures, f^assification of students by handi- 
capped, disadvantaged, LEP, racial /etPinic,-ancr sex. ^fc addition, 
student undup'licated headcounts in occupational programs are a>so 
reported. -The data in the VEDS are aggregated across localities, 
and states to form a national picture (Section 163(a)(1)). 
Quality of programs and services provided to LEP s.tudents is 
■ rarely accomplished. According to Smith and Holt (1980), the evalu- 
^ ation procedures' for the measuj^efnent of student achievement consti- 
tutes the least developed component of state vocational education ^ 
\ evaluation systems. Smith and Holt [1980) state *the following:- 
"Only one-fift^rof the states had in 1978 procedures for the as- 
sessment of student achievement; those. procedures were mostly in 
the development or pilot^testing state" (P. 34). , 

Mainly through the impetus of Pubjic Uaw 94-482 which identified 
persons of LEP as a national priority, states began developirrg , 
programs and ^services to meet .tl}e needs of LEP students. However, 
the development and evaluation requirements specified in the law 

r ^ 

(P.L. 94-482') required knowledge and e;<pertise which many state 
personnel did not appear to have. - ' ^ ^ . 

Critical Views and Concents ^ ^ 

W 

Interviews were conducted with a variety of individuals. Persons 
interviewed weVf members. of the S€^te Advisory Council for Vocational 
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Education, staff of the DM^E^ local administrators, local instructors,- 
local guidance and counseling personnel, students and community 
members. The total number of interviews ^held with "different indi- 
viduals totaled forty-seven (47). Some persons were interviewed 
twice for the ^jurpose of amplication, clarification or verification 
of themes and issues. The analysis of the data gained through -the ' 
interviews was an ongoing process. ' " / 

Four themes were identified by the interviewers as being 
major issues or concerns Velevant to the evaluation of vocational 
programs or services serving LEP students. The interviewer was-** 
not attempting to generate standardized stimuli, such as test items 
-.or questionnaire items. The principal stimuli was considered to 
be those issues or concerns whicli were most germane or natural to 
the interviews. The four main thehjes emerging from the interviews 
\^ ./are as follows: ' . ■ , 

^ ' A, Strenqths and weaknesses of the on-site visitation, • ^ 



B. Changes, occurring in the areas of programs and ser- 
vices aimed at LEP students as a result'*of the on- 
^ site evaluation. * * 

Ci • Expertise and composition of on-site Team Members, 

D, Possible Changes of the on-site evaluation process 
with respect to evaluating programs and services 
aimed at LEP students, 

^ The. discussion and conclusions drawn from the data collected 

via the interviews will be presented according to the four main 

themes. 

- A.- Strengths and weakness' of the on-site visitation^- 
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strengths of the on-site evaluation which were identified 
are as follows: the positive effects on team members, the^,. 
positive impact on LEA*s, beneficial and timely evaluation 
reports-, opportunity by LEA personnel to express opinions, 

"and the comprehensiveness of the evaluation process. Jhe 
issues related to the weaknesses of the on-site evaluation 

.a-r^-as follows:' th§ composition of the .team members 1 ikel i- 
hood of friends evaluating friends, lack of .LEA preparatiop 
prior to on-site visitation, lack of team member preparation 
prior to on-site visitation, and the inlbility to evaluate' 
the qualfty of instruction and programs via the ofi-site 
process. 

The strengths and benefits of the on-site evaluation pro- 
cess are'generally seen as* far out-weighing it's weaknesses. 
The TPS was perceived by local .administrators. Team Leaders, 
and the staff of the DAVTE as being ttie "moving force" which 
•a'Ssisted vocational education to develop and maintain a high 
level of quality throughout thfi State of Illinois. Most 
weaknesses of the system were perceived as minor deficiencies 
of an otherwise excellent syswn. The impact. of the TPS on 
LEAs over the last ten years, was seen as most productive. 
However, Team Leaders and local administrators whiqh had 
extensive experience with the TPS expressed a concern relativ 
to lack of follow-through on recommendations by the £)AVTE. 
This lack of follow-through on recommendations both by LEAs 
and the DAVTE may very well affectHh^^ffectiveness of the, 
TPSo'n the future. 

a-oi \ 

I 
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Changes occurring in the area of programs and services .aimed 
at LEP •^students as a result of the on-site evaluation. 
The interview and questionnaire data, concerning changes that 
had" occurred in LEP program^ and services as a result =of the 
von-site visitatioTi did not reflect variability. The data con- 
sistently reflected a paucity of changes as a result of an 
on-site visitation, The^rcity of changes wh-ich were attri- 
buted to the on-site evaluation are most likely explained by 
(1) the lack of emphasis placed on LEP students by the TPS, 
and (2.i.the lack of awareness concerning LEP student needs 
by the on-site evaluation team.^This deartfyof emphalfs , 
and awareness on LEP students, appears to significantly 
affect the quantity and quality of conclusions, recommenda- 

4 

tions, and suggested improvements provided in the final orT- 
site evaluation report. 

. The content related to LEP issues and concerns- found in ' 
final on-site evaluation reports were extremely scarce. In 
reviewing final evaluation reports, concJ,usions and recom- 
.mendations generally addressed whether handicapped, disad- 
vantaged or lEP students v^ere receiving additional services. 
The quality or impactrOf these services vyere rarely addressed. 
Acldittonally-, suggested improvemerits rarely reflected compre- 
hensivei or in-depth improvements towa'rd LEP programs and 
services, Those changes "occurring in LEAs according to 
tnterviews, were the results of lEA personnel interests and 
efforts tn the quest to'better serve LEP' students. 

102 




Expertise and composition of on-site team members. 
The interview and questionnaire data regarding the effects 
that evaluation team member expertise hav^ on the evaluation 
of programs and services aimed at LEP students were varied. - 
The data -ranged from the effects being unsubstahtial to being 
critical in evaluating these type of programs and services. - 
Persons who perceived the effects on the programs as being, 
urri^bstantial, also viewed the concept of teamwork as being 
pr(oductive and beneficial. Team Leaders were often times 
viewed as the key to the comprehensiveness of the on-site 
evaluation. Team Leaders are general Ij^ expected by the DAVTE 
to be cognizant of issues and concerns relative to LEP stu- 
ctent programs and services.*-. 

Selection of qualified team members was seen as essential 
and advantageous for a comprehensive evaluation,- Local edu- 
cation agencies which were serving LEP students and were 
being evaluated by .an on-site team which did not have an 
experienced special needs team member, were perce1v.ed as not 
fully benefiting from a compr^ensive on-site -evaluation. 
Possible changes-'Of the on-site evaluation process with 
respect to^evaluating programs and services aimed at LEP 
students. , ' 

The interview and questionnaire data related to ppsrsible 
changes of the on-s^te evaluation process regarding LEP ^ 
stud^nt^ were diverse. Most persons interviewed focused on 
three changes. They "were -as follows: the jcomposition of 

' 403 ' • . 
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the team, the interview tasks in the Team Member Handbook, 
and the role of the BAVTE in assisting LEA's implement 
recommendations found in the final evaluation report. 

The changes suggested in the interviews indicated the 
lack of emphasis by the TPS on LEP issues and concerns and 
also the lack of awareness concerning LEP student needs by 
on-site evaluation teams. Possible changes which were sug- 
gested, attempt- to increase the emphas|s pn LEP student 



programs and services by adding interview tpsk to the Team 
Member Handbook and also by insuring that team members have 
the necessary experience and knowledge relative to LEP ^pro- 
grams and services. Persons \iq^erviewed realize th^ lack 
of expertise artd information available regarding LEP stu--. 
dents and appeared to place the responsibility^ wi^th the DAVTE 
for /increasing the emphasis- on LEP student programs and services 
arjd the awareness of LEP student needs. This 'could be ac- 
complished by disseminating the most up-to-date developments 
and strategies on how to serve LEP students -via publications, 
ip-service workshops, consultant services and through an on- 
site evaluatipn which could cultivate, 'sensitize and encourage 
LEAs to better serve LEP students'/ 
The Extent and Nature of the Irrtpact ■ 

Several data gathering, techniques were used to collect data re- . 
garding the extent and nature of ttie TPS impact on local education 
agencies that were evaluated in school year 1979-80, The dajta were 
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collected through a questionnaire and formal and informal interviews. 
The questionnaire which was mailed, to 33 LEA vocational education 
direc,torr was designed to provide some quantifiable measures of the 
effect the on-site visitation had on programs and services provided 
to LEP students.. Following the Questionnaire on the Effects of th^ ^ 
TPS, interviews were cofiducted with-personnel of three Selected ^• 
LEAs, Team l^eaders, LEP students, and regional vocational adminis- 
trators. The purpose of the interviews were to acquire a further . j 
*in-depth perspective of the impact of the on-site eve^ation upoa 
programs and services^ provided to LEP' students. 

Results from the questionnaire -indicated that over 76% of the 
^respondents rated their overall vocational education program in' 
their district as being "above" or "well above" average. Data al^o 
indicated that a minimal percentage (0.06%) of LEA administrators, 
vocational instructors and guidance/counseling personnel were able" 
to communicate at Jleast minimally with LEP students in their native 
language. Furthermore, the data indicated that 76% of' the LEA 
staff had little or no in-service training concerning issues* and 
concerns of LEP students within the last two yeans,' 

*Data. from the questionnaire regarding the itnpact of the on-site ^ 
evaluation were varied. Nearly 62% of the responding administrators 
indicatisd that the LEAs had gotten some good ideas on how to improve, 
programs and services aimed at LEP students. However, over 66% of the 
respondents in'Bicated^that the changes that were being made in pro-* 
grams and services; for LEP student were not the direct result of 
information provided by the on-site ^Valuation rep*Qrt, Moreover, 57% 
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indiccited that the on-site evaluation- process had not assisted 
their LEAs 1n becoming aware of resources or services designed to 
improve programs and services aimed at l£P students. 

Ttfe data acquired through interviews depicted rninimal impact 
on programs and services aimed at LEP students by the on-site evalu- 
^atXeifi. Changes that were made in the improvement of programs and 
services were perceived as a direct result of the effo'rts of LEA 
personnel and were not. direct results of the on-site evaluation; 
OBSERVATIONS AND RECOMMENDATIONS ^ ' ^ \^ 

The DAVTE on-site evaluation pr'bcess was discovered to i\e a 
formidable positive factor in the overall maintenance, growth and 
quality ^f vocational education programs and services in Illinois. 
This study revealed, however, that the on-site evaluation process 
was having alninimal positive impact on vocational education pro-^ 
grams and servi'^es serving LEP students. Whil^^vocational educ|[tipn 
pro grams. aimed at LEP populations are a relatively recent educational' 
forum,-. very little is known about. the operation of t}iese program 
the tnstructiona'Upr^ct^ces, and tifie effects on trainees in - • 
111 inots and other states, . 

The TPS in accomplishing it's three general goals of cjev^ptng, 

planning and insuring accountalnTTEy^or vocfational .education, 

should sey*ve to make persons involved in^ vocational eoucatiion more 

aware of lER issues and concerns. Furthermore, the on-site evalu^ 

atfon process in evalu^ng services provided to liEP papulations 

should also serve to increase the awareness and^emphasis 'on XEP 

students. within the state and local education agencies. In addition 
/ 

106- 



i 



to obtaining this increased awareness of and emphaj4s on LEP stu- 
dents, formative program data would assist LEAs in making appropriate 
changes anh improvements in programs and serv'ices provided to their 
LEP population. In view of the minimal positive impact the on-^site 
evaluation , is having on programs and services serving LEP students, 
recommendations are provided to strengthen an already excellent 
TPS. These recommendations r^esent a host of vjews Qollected in 
the study; they are global in nature and are not presented in ^ 
prioritized .fashion. ' ^ . ^ 

The^,DAVTE should 4:ontinue with the on-site evaluation 
with -some modification. 

The DAVTE should evaluate the antire TPS to. i?nsure quality ^ 
of programs and instruction of/alT Vocati'onal education / ^ 
in the State of Illinois. ^ \ ^ . ' ' ^ " 

The DAVTE should establish procedures tor,ident1fy indi- ^ 
vidua-ls who can serve as Gonsul tants to t6e. DAVTE and. 
LEAs relative to evaluation ofi LEP programs and services. . 
These individuals could 'provide several types services; 
serve as Team Mepibers, provide inservice train-yfig to 
Team^ Lead&s,. Team Members, and LEA personnel. Thj^ 
could also assist in J|entification apd development of 
criteria and guidelines for quality programs and ser- 
vices provided tOj4.El^ students* . • " ' . 

The DAVTE should d§velop and/or identify Existing re- 
sources which, would assist LEAs in developing and^^valu-. ^^^^^ 
atfng programs and seryites to LEP. students. * ^ \ - 

T?ie DAVTE should develop criteria and guidelin^ to 
assist Team Leaders and Team^ Members in evaluating 
^ programs and services aimecl at LEP students. 

The DAVTE should increase Jbhe utilization of data^ * 
already , being collected via the student/ employer 
foH low-up studies and the VEDS reporting system* 
relative* to LEP students. Utilization may*include/ 
studies and Special reports concerning effects cfr , 
vocational education programs fhd services on*^LEP 
J populations. ' 
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The DAVTE'AshDuld modify or supplement the Team Member 
Handbook, the School -and Community Data Form, < the stu- 
dent and faculty Preliminary Evaluation Instruments /to 
solicit specific data relative to quality and impact of 
programs and services provided to LEP students/, . 

Jh€ DA^TE should increase and strengthen it*s follow-up 
activities following on-site evaluations. This activity 
wi)l insure and-iissist LEAs with the/implement'ation of 
the , recommendations made by the on-site evaluation team. 

The DAVTE should 5tudy the benefits of incorporating a- 
self-study component into the TPS. ^ 

The DAVTE should consider utilizing of a separate evaluation 
prapess (separate from the TPS) which would provide an 
•in-depth perspective 'on programs and services serving 
KP students. Poss^ible evaluation alternatives could be: 

1. tdentify a cadre of consultants which are knowledgeable 

.in the^rea of LEP students, and contract these consultants 
' to conduct an in-depth evaluation of programs and 
.'services provided to LEP students 

2. J§Ie€t LEAs that are serving communities with a sizable 
LEP population and have a consultant from the DAVTE, 
the Regional Vocational Administrator and an LEP 

, ^ consultant cpnduct an in-depth evaluation of the programs 
ar]d service?. provided to LEP students ^ 



3. ' Once every four ypars the DAVTE should require LEAs that 
are serving LEP students,- to conductJuiternal i^-depth 
evaluations with regards to the programs and services 
'being provided to LEP students. LEAs could possibly 
^ , acquire assistance from the ijadre of^ consultants 
/ identified in alternative,! 

The DAVTE should sponsor further studies on bdw to provide 
. comprehensive and. quality vocational .education programs and 

services tcr LEP students. In addition, studies should focus 
• on how to determine the effects "^f these programs and services 

on LEP students. ? / 

Local education agencies should^ utilize' the Locally Directed 
Evaluation Materials provided by the DAVTE to evaluate their 
own programs and services aimed at LEP students. Jhis locally 
directed ev^^tion process would assist LEA^ personnel in 
becoming more aware of LEP student needs and consequently 

^change or improve the programs and services being afforded to 

' LEP students. 



108 

'y Ho 



Local education agencies shQiJd develop procedures for 
identifying LEP students in Vocational education. This * 
lack of procedure's is consistent' with the findings reported . 
by Cheaney (1981). _ ^ 

Local education agencies should be sapportive of the efforts 
of .vocational- instructors, VESL/ESL instructors and the 
.guidance/counseling staff in serving the needs of LEP students 

LEA's should utilize whatever resources or services that are 
made available to tHeip by DAVTE concerning LEP student 
.programs and services. 

Universities should offer^off-campus c^ijjrses for local 
personnel on planning and evaluating vocational education 
programs and services for. LEP students. 

Universities should encourage further research on the>l.anning 
and evaluating of vocational education^ programs and services 
for LEP students. 
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APPENDIX A 



• QUESTIONNAIRE ON EVALU/CTION PRACTICES 

Your cooperation is requested in completing, the attached questionnaire conceriiing 
evaluation practices being used to evaluate vocational education serviced aimed 
at limited English proficiency (LEP) students* This study is being conducted 
by the Research, Evaluatidn, and Program Imj)rovement for Limited English Proficiency 
Students in Vocational Education Project In cooperation with the Illinois Office 
of Education, Department of Adult, Vocational and Technical Education,, 

The purpose of this questionnaire is to identify evaluation practices being 
used to meet the requirements set forth by Section 112 of the .Education Amendments 
(Public Law 94-482). The information will expand our knowledge of the purrently 
used evaluation methodologies and aid in provi>4wg a foundation for the development 
of a system that adequately evaluates services provided for LEP students in all 
st^te^inograms. 



Please return the completed questionnaire by lOecember 12, 1980 to: 

L. Allen Phelps 
Department of Vocational 
^nd Technte^n Education 
' Opiversity of Illinois 
805 W, : Pennsylvania 
^ ' ' Urbana, iL 61801 • * 



If you have any questions call us at (217) 333-2325, 

Your prompt response to this request will be greatly appreciated, 



PLEASE CHECK THE ANSWER YOU MOST AGREE WITHAND PROVIDE BRIEF DESCRIPTIONS WHEN 
NECESSARY. - 



}7 



D^es jour state have a ^separate' evaluation process for evaluating vocational 
prbgrams serving Limited English Proficiency (LEP) students? (separate 
from regular vocational education evaluation or other state evaluation 
systems) • ^ * 

YES - ,' NO J 

If the answer is YES, please provide a brifef description. 
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2. Your state is effective in meeting the requirements set forth by Section 112 
.(Title II Education Amendments of 1976) in evaluating aV[ vocational education 
^programs. . . * 

Strongly Agree 

. Agree 

Neutral j ^ 

Disagree 

>^ 

Strongly Disagree ^ 



i 



Your state. is effective in evaluating services and programs aimed at LEP' 
students, 

» J, 

^ Strongly Agree 

Agree ' ' 

Neutral 

Disagree 

Strongly Disagree - * 



4. Your state is effective in utilizing evaluation results in improvement 
of services and programs aimed at LEP students. 

Strongly Agree ^ 

Agree ' , 

■'Neutral 

Disagree 

Strongly Disagree. _^ • , 

^5. The evaluation efforts in your state need to be revised to meet the needs 
of LEP students." 

Strongly Agree ' ' * . * 

Agree ^ ' • ' 

Neutral 

Disagree 



Strongly Disagree ^ 127 



Do you f^^your state's follow-up stil^ies of LEP students need to be improved? 



YES '• 



NO 



If ith6 answer is YES, please provide' a br-fef description of the improvements 

\ . ^ \ ■■■ 

you fefl would be necessary. . 



Do you feel state's employer follow-up Studies of LEP students, need- 
to be improved? 



YES 



NO 



J 



If the answer is YES, please provide a brief description of the improvemen.ts 
you feel would be necessary. . ' • 



Please send us any material or literature that describes 

evaluation process or that supplements your^answers fn the questionnaire. 

Please notify us prior to mailing if there is a charge for these materials, 



Would you like to' recei.ve thexresults of this study?- 

YES ■ NO ^ 

Please fill in your name, address, awi phone number for possible, future 
communication regarding more detailea explanations of * ' evaluation 
practices^ . • 



r 



THANK you FOR YOUR -COOPERATION AND ASSISTANCE. 



I 
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A STUDY OF THE EFFECTS OF THE ILLINOIS SYSTEM FOR 
EVALUATION OF VOCATIONAL EDUCATION UPON THE PROGRAMS 
'SERVING LIMITED ENGLISH SPEAKING STUDENTS 



1 » 
-Your cooperation is' requested in completing the following questionnaire 
concerning- programs and services provided to Limited Engl ish Proficiency 
(LEPJ students. The focus of this questionnaire i's to determine. (the it 
]'"Pw$*x°^ Department of Adult, Vocational' and 'Technical Education 
(DAVTE) on-site evaluation conducted iji fiscal /ear 1979-80. The study 
IS being conducted by the University of Illinois atiUrbana-Champa''ign in 
cooperation with th6 Illinois state Board of' Education and the DAVTE. 

Specifically, this questionnaire atteiripts to identify evaluation practices 
that have effected programs and services aimed at lEP students. Tfie 
information gained will expand oUr knowledge of the impact of ithe currently 
used evaluation process. ^ f 

» \ ' , 

Information obtained in this questionnaire will be kept ^n confidence 
Please respond to all items based on your expgV-ience 'and test judgement. ■ • 



DIRECTIONS 



fhere are no right or wrong answers for the items in this questionnaire. 
The valued of this study depends on your best judgenients. Please attempt 
to respond to. all items: Answer each question by airaling the- answer code 
unless otherwise instrusted. '(Use the Don't Know (DK) only if you simply 
oatmot use another response category. ) . 



1. What is your position title? 



2. Have you ever, served as a member of a DAVTE on-site evaluation team' 

Yes- ; • No 2 ' ' 

If your response to question 2 is Yes, on how many teams have you served? 
teams. 
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Do' you speakeanother language other than English? 

■ Yes 1 No , 2 

If ybtfr response. to question 3 'is'Yes, please Tist your language(s) and 
note your speaking ability. 

^Lai^guage^ . -Rair "Good ExceTlent 



How would you rate the vocational education program in your di*strict? ' 

* Well above'average ; 1 

Above average 2 

Average * , 3 

Below average ^ ' C 

Well below average 5 ♦ 



Rate the cooperation between vocational faculty teaching LEP students and 
the guidance/counseling personnel. 

High cooperation ; •! / 

Averahe cooperation 2 ' - / 

Low cpoperation , \ 3 / 

R^te the cooperation between v&c&tional faculty teaching L'EP students, and 
the vocational English as a second'^a^anguage (VESL) faculty br the English 
as a second language ^ESL) faculty, > , , / 

-I ' 

Hjgh cooperation » 1 / 

Average^ cooperatipn .^^A^s^ 2 . . , 

.Low cooperation ^ ^ , ^,,3 ^ ' • 

To the best ^f yo^ir kiftwledge rate the cultural sen5itivity to LEP stiidents« 
by the'following staff, (e,g, the ability to acknowledge and appreciatie the 
reality of differences within and among ethnically or culturally diverse' 
students) , ^ ° . * ^ . , , \^ [ ^ 

J ' ;^ > . / ^ 

\ ^ ' Cultural Sensitivity . 

a ^ *^ High ^ Ave/age Law ' 

'VoMtionarinstructors 1 r ,2 ' " 3 v 

^Guidance/Counseling staff 1 .2 3 ° 

'VESL/ESL instructors • .1 2 3 

Admin istratgrs , ® ^ Ir^^ 2 3 

, ■ i ■ ■ 
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8. To the "besir of your knowledge hdw many o-f the following staff are able 
to communicate at least rainimal/iy with LEP students in theiV native 
language?^ (e.g. 1 of 3, 1 of p , - ,, 

^ • , — > 
Language(s) 



of 



■ of 



of 



of 



Administrators ♦ 
Vocationkl faculty 
•VESL/ESL faculty ' 
Guidance/Counseling staff 



9. How often has staff recently (within the last 2 years) Attended in-service 
presentations or workshops conce»(ning issues arid concerns of LEP students 
in vocational education programs \ind services?. • . • 



Extensively 
Moderately 
Little 
None 



1 
2 
3 
4* 



• 4 



The^ remaining questions foous on the on-site evaluation oonduoted in 'your 
distrvo^ ^n school yecxr 197^-80. ^ 



— r- 



10. What persons invol-ved with LEP students were invited to thg on-site 
evaluation exit interview?^ (Circle Yes or Np) 



YES 



„ No 
2 

" 2 
' . 2 
2 

' 2 
2 
2 



Administrators ' 
Vocational in,structors 
VESL/ESL staff 
Guidance/Counseling staff 
Parents 

Board members ' i 
Others 



How Many Attended? 

, c 



11. What persons involved with UEP students received the final rep6rt of the 
on-site evaluation? (Circle YES or NO) . * 



YES 
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NO 
2 



■2 
, 2 
2 
2 



Administrators 

Vocational faculty 

VESL/ESL faculty 

.Guidanie/Counseling staff 

Board members 

Parfenls 

Others 



Estimate, % who-^ 
received report - 

% 

% 

«% 

% 
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12. To vrtiat extent are the persons involved withal students familiar with 
the information included in the evaluation report concerning LEP student 
programs and services? 

Extensively Modera.t&Ly Little ^ Ngnq 

Administrators 

Vocational faculty ^ 

VESL/ESL faculty ' ^ 

Guidance/Counseling staff 

( Board members' ' ' * 

V"^ Parents ^^^^^^[^ HZHZ ^ 

^ Others _j ^_ 



. L _ 

For each of the following statements please indigafe whether you strongly agree^ 

agree^ disagree or strongly disagree. If you simply cannot respond tmvn.temj 

qircle 9 for don't know. Circle only one response for each statement. i 



13. rlhink^the DA\(tE on-site evaluation helped us to improve our entire ^ 
vocationa^l education program. 

Strpngly agree 1 

Agree ^ ; 2 

Disagree 3 

Strongly disagree 4 

Don't know 9 

14. During the DAVTE on-site eval uation^our LEA got some good ideas on how to 
improve the vocational educatipn programs and services aimed at LEP 

- students. 

, Strongly agVee 1 

Agree ^ 2 

Disagree 3 

Strongly disagree 1 4 

Don'.t know I -9 




15. T6 the best of my knowledge the DAVTE evaluation team members who interviewed 
our LEA were knpwledgable in the area of LEP students. 

\< * • » * ^ 

Strongly agree ^ ' 1 

A^ree 2 

Disagree ^ 3 

Strongly disagree 4 

Don't know 9 ^\ 
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Through the on-site evaluation, our LEA became aware of resources and 
services that will/are being U5ed to improve pro^grams and services for 
LEP students. 

Strongly agree 1 

Agree 2 

Disagree 3 

Strongly disagree ' ^ 4 

Don't know '9 , 1 

To what extent have these resources beerv utilized? 

Extensively • ; <,'••••! 

Moderately 2 

Little ,'3 

Not at air 4 

The district has requested DAVTE consultant services to assist with 
implementing the recommendations regarding LEP students. . 

« 

Yes ; 1 / 

No 2 "\ ' 



If^your response to question 17 is Yes, pl6ase describe the services 
received. ^ 

\ . ' - 



rn your opinion, was Adequate time spent by the evaluation team in 
examining the programs and services related to LEP students? 

Strongly e^gree. 1 

Agree 2 

Disagree 3 

Strongly disagree .4 " ^ 

Don't know .,>v. 9 
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19. Many of the changes that will/are being made in our vocational education 
program and services for LEP students ha\?e been the result of information 
^provided by the on-site evaluation report. 



Strongly agree 1 

Agree 2 

Disagree ^ ' 3 

Strongly disagree 4 

Don't know ..... .\ : 9 ^ 

Please list any changes. 



20. Please add any other comments, perceptions, recommendations, 'etc. , 

regarding the evaluation of vocational education programs and services 
aimed at LEP students, 



V 



THANK >0U VERY MUCH FOR" YOUR COOPERATION 

t 

Juan C. Gonzalez 
Pepartment of Vocational ' . 
and Technical Education 
. ' - University of Illinois . 

' 805 W. Pennsylvania Avenue 

UrbSFha, IL 61801 
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.Three Phase System for Statewide Evaluation s^^^^ 
of Vocational Education Programs (TPS) 

The goals of the TPS^re: ^ 

1. To promote and assist in the developmenV-ofr^trality voca- 
tional programs in local education agencies; 

2. To foster maximum utilization and cTccQuntabil ity* of state 
and. federal funds allocated to local vocational programs. 

3. To provide the Department of Adult,* Vocational and Technical 
Education of the State Board with necessa^ data for state- 
wide planning of, vocational education^) 

The TPS was develope'd to emphasis the importance for a continuing 
mechanism for improving local agency vocational education programs and 
services. The TPS 'requires individual commitment at the state and 
local leveTs to evaluation as a vehicle for program improvement. The 
TPS has three phases: 

Phase One Development of Local Plans 
Pha^e Two Review and Approval of Local Plans 
Phase Thrfee On-site Visitation * * 

Tjie following descriptions are drawn from the Evaluation Handbook for 
Team Leaders , by the Illinois Stat? Board of Education (1980). 

Phase One ' 

Phase One focuses on internal evaluation and planning of the 
total vocational education program at the loca4 level. Responsibility 
for this Phase rests v/ith local education agency personnel along with* 
members of the board of control and advisory groups who conduct 
locally directed evaluation activities' and hold planning sessions. 
The outcome of Phase One is a document entitled the Local Agency One 
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and Five Year Plan for Vocational Education. The Local Plan describes 
the components and operation of the total a§enc^ vocational education 
prograirrf 

Phase Two 

Ph«e Two provides for a review and analysis of the proposed 
total program of vocational education. .This is accomplished through 
the review of Local Plan by a cadre of State Board staff who determine 
its appropriateness in light of local needs and resources. The out- 
come of Phase Two is an approval status for the One and Five Year 
Plan.- A Plan may be "Approved," "Conditionally Approved," or "Not 
Appro^Aed^" 



Phase Three 

Phase Three of the System provides an external review of the 
total vocational educjition program by a team of persons from outside 
the local agency. This review focuses on the extent to which the 
educational agency is meeting student, employer and community needs 
as reflected in the agency's own Local Plan. (Illinois State Board 
of Education, 1980, p. 4). • ^ 

The TPS emphasized the evaluation of the quaVity of the total 
vocational program and its component parts. Six/ program components 
have been identified as essential to the , success of the vocational 
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education program. The following descriptions of the six components 
are drawn from An Overview for Team Members by th^ Illinois State \. 
Board of Education, (1981). , • . f 
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I'LANNINt; AND EVALlJAIION , 
Ivulonco is gathered by the team to determine the afJcf|U<uy of 
ihe'local educaiton agency's system for establishing total program, 
goals, identifying needed vocational programs, and designing in^truci.ion 
Additionally, the local education agqncy's system ^of evaluating both 
the processes of instruction and the outcomes or impact of programs 
is assessed. 

VOCATIONAL PROGRAMS 
The team analyzes the guality o'f each vocational program through 
interviews and document review to as&<^ss the availability of programs 
to all students as well as program scope and course sequence. Addi- 
tionally, the team will assess the adequacy of internal resource 
utilization within various programs. s 

STUDENT SERVICES * 
The various support or ancillary services provided to vocational 
students are reviewed by the evaluation team. Services Such as 
guidance and counseling, -placement, and student testing are focused 
on through the interview and document review process. Services for 
special populations such as the disadvantaged, limvted English - 
proficiency, handicappe^iffTnSMty' groups and women are also included 

X 

in this component. • ' 

PERSONNEL ^ 

Ik 

The team reviews personnel qualifications in light of personnel 
development needs and analyzfes the type and extent of participation 
in various personnel development activities sUch as professional 
organizations, workshops", and ^courses. 
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PROGRAM. MANAGEMENT ^ 
The team examines the way vocational programs are managed. This 
includes review of both the administrative structure as well as the 
operational effe^i\^eness of the structure. This component | includes 
management at all levels of the local education agency that contri- 
butes^. tljje delivery of instruction and services to vocaj|ional students. 

QOMMUNITY RESOURCES . 

The team focuses on this component to ideteriyne to what extent 
the local education Agency has involved community representatives and 
utilized coimn^ity resources in the development, operation and evalu- 
ation of vocational programs. The use of advisory committees, co- 
opera^on with other educational and service agencies and cooperative 
education pursuits are examples 6y areas to be reviewed in tMs component. 

In order to in^pre at thorougti on-site evaluation of a local 
agency's vocational 'education progi^am, the team wi IT consist of indi- 
viduals varying in areas of expertise. Team menbers are selected from 
outside the jmnediate locale of the local education agency. Team ' 
members representing the following thre^^ areas are selected for each 
evaluation. They are: 

1. Business, industry, or labor representatives who have a 
\i^cern for and an interest in vocational-education. 

2. Practicing educators who^-hjive d'anonstrated teaching, adminis- 
trative or student 'support service expertise* in vocational 
education. * ' ^ 

3. Advanced students and graduates of vocational program?. 



The number of team members in eacfr of these three categories depefnds 
upon size and the characteristics of the agency being evaluated^ nPHe 
following is a typical -schedule for a four-day visit^as described in. 
. the T^am Leader Handbook , 
Previsitation Activities 

Day One Team Leader Meeting with Ageifcy Representative 

Team Member Orientation - 

Day Two — Staff-Team* Orientation 
Interv-iewing 

Team Conference (Report writing) 

Day Thr*ee Interviewing ' ^ 

Finalize Evaluation^ Report 

Day Four Summary Conference 

For a five-day visit, Day Two and Day Three will be essentially the 
same. Day Four will, be as Day Three above, with the summary conference 
on Day Five* 

, Materials are provided to the team members prior to the on-site 

visitation. The local agency and the DAVTE send each ipember copies 

i 

of: 

1. The Local Plan which was prepared by the local agency staff 
* (tn. Phase I) and approved by the State Board Staff (in Phase 

II), The Local Plan describes wfiat the agency is. doing f^^r^ 
vocational students. 

2, The Team Member Handbook contains primary goals and interv)ew 
, tasks/ The. preliminary evaluation information gathered rrom 

students and, faculty will be provided by the tearry leadc 

These data related to the interview tasks are used to 

,» * 

formulate conclusions for the evluation report. 
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The end product of. the on-site visitation is an evaluation report 
reflective of the local agency's total vocational education program. 
The evaluation report contains conclusions, recommendations and 
suggested improvements which address the specific needs of the agency 
being evaluated. The report is duplicated by the DAVTE'and returned 
to the LEA in suf/ficiept quantity for distribution to faculty, the 
board of control and other persons involved in the program improvement 
process. The LEA. must respond to each recommendation in Section B 
(Progranj Improvement Plan) of the, next One and Five Year Plan 
submitted the following Spring, 

Various type of data gathering instruments are used in the on- 
site visitation. These instruments include Student and Faculty 
Preliminary Evaluation Instruments (PEI), the School and Community 
Data Form, tbe Team Leader Questionnaire, the Student and Employer 
Follow-up Surveys, and the Team Member Handbook (which provides 
Primary Goals and Interview Tasks for the team members). 

The results of the Student and FacuTty PETs and the data from 
the Schoal and Coimiunity Oata form are given .to thfe team members 
during the team orientation and- are also incorporated int-o ihe Team 

M6mber Handbook, This data aides the team members in conducting 

\ ^ ' ' ' 

interviews and writing the evaluation report. 
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OFESSIONAL 
STARF FORM 



DIRECTIONS FOR l\^RKING 
Use tt2 pencil only-erase cleanly 
' * Mark only one answer per item. 



PERSONAL nviFORMATlON , 

1. I am. 

O Male , O Female 

2. Your present primary position is? 
O Administration O Instruction 
O Guidance 

3. If instruction, in which vocational area 
/are you most involved? 

O Agriculture 

O Business, marketing, management 
• O Health 

O Home Economics 
O Industrial 

O Cooperative Education ^ 

4. How many years of experience in voca- 
tional education have you had (teaching, 
guidance, administration)? 

QrO ■ 4 years O 13 - 16 years 

O 5 • 8 years O 16 or more years 

O 9 ■ 12 years 

5. Hovy many years of work experience out- 
side of education have you bad in your 
area of teaching? 

O None O 9 > i1 2 years 

O Lei^than 1 year Q Mor^a than 12 years 
O I 4 years Q ' am not presently 
O 5 ■ 8 years teaching * 

6. Are you presently a member t>f a profes- 
sional vocational organization related to 
your field, such as the Illinois Vocational 
Association? o 

O Yes - O No 



PROGRAM INFORMATION 

7. Are there written and measurable objec- 
th/es for the fbllowtngr 
a The programlsK^ in which you teach. ' 
administer, or provide guidance?* 
O Yes O No 

b. The' coursels) in which you teech. ad* 
1 /( 4 nfiintster. or provide guidance? ^ 

ERJC I II 1 1 I I I I I I I J 



8* How would you describe the locally c 
acted (self) evaluation system for voca- 
tional education in your agency? 

. O A formal evaluation system exists 
O Some informal evaluation activities exist 
O process^of being developed 

(3 No evaluation activities exist 

9. If a formal evaluation system exists in 
your ^ency. are the results utilized in 
program planning and development? 
O Yes O No 

Q No formal evaluation system exists 

10 Have you had the opportunity to make 
contributions to the development of 
your local agency One and Five Year 
Plan for Vocational Education? 
O Xes O No 

O I vvasn't here when 
the Plan was developed 

11. If your agency previously had a State 
Board of Education vocational program 

^ evaluation, did you receive a copy of 
the evaluation report? 
O Yes O No 

O ' wasn't here then ^ 

12. If you received a copy of the evaluation 
report, was the report used in making 
prograrti improvements? 

O Yes O No 

O I wasn't here then 

13. To what extent are you involved with 
the vocational advisory council/ conv 
mittee(s)? 

O Extensively O Little 

O Moderately . O None 

14. Does your agency have a system for 
identifying, disadvantaged, handicap- 
ped and limited English proficiency 

jgjdents? 

C>Yes Ono ' 

15. Do you have stu^^lnts in your class(es) 
who are: a 
a. Di^dvantaged? 

O Yes O No OAJncerlam 
b Limited in English proficiency? « 

O Yes O No ' O Uncertain 
c. Handicapped? 

OYes -' ONq O Uncertain 

I I I I I I I I I I I I I I I 



1 6 Are additional services provided by 
your agency for disadvantaged, limited 
English proficiency and handicapped 
students (other than special education)? 
a Disadvantaged? 

O Yes O No O Uncertairi 
b. Limited in English proficiency? 

O Yes O No O Uncertain 
c Handicapped? 

OYes ^ <0 No O Uncertain 

17. How would you ra|e the local board's 
support of A/ocational education? 

O High O Average O Low 

18. How would you rate guidance person- 
nel's support for vocational course and 
program offering^? 

O High Q Average Q Low 

19 Do guidance personnel and instructors 
work togetherdn 

a Identifying student's 
needs and interests 
% Q Extensively 
Q Moderately 
O Little 
O None ^ 
b Student Placement , 
O Extensively 
O Moderately 
O Little 
None 

20 How would you rate the working rela- 
tionship among vocational faculty and 
agency adniinistrators? 

Average O Low 



re the vocational courses offered by 
I your agency sequen|ially structured in 
to programs? 

Q ^ courses are part of a sequence 
, Q Most courses are part of a sequence 
■ O Some courses are part of a sequence 

O Courses are not sequentially structured 
j 23 To what extent have you been involved 

in articulating vocatibnai programs. 

such as discussions with other pro- 

gram personnel, both within<«nd out- 
. side of your agency^ 
* ' O Extensively 
' O Moderately »- 

O Little 

0 None . . 

24 Do you encourage students of either 
sex to enroll in your vocational courses? 

! O.Yes Ono 
I 

I 25 Are handicapped students encouraged 
to enroll in vocational courses? 

O Yes O No 

26 Have you participated in vocational in- 
service a^ivities which emphasized 
equal educational opportunity for all 
students? ^ . ^ 

O Yes O No 

^ Does an effective public relations sys* 
tem exist for the total vocational educa- 
tion program? 



O Yes 



Ono 




2V How would you raXS the adequacy cH equipment, 
facilities and safety in your agency? r / 
• a Equipment 

O Up t6-date O Fa'f O Obsolete 

b Facilities ^ . 

O Up-to-date Q ^a»f O Obiolt*te 

c. Safety (housekeeping, eye protection, venti- 
lation, etc ) ^ 
O Up to dale O P.ur O Obsolete 



00 NOT 
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/lENT OF ADULT. VOCATIONAL AND TECHNICAL 
CATION-ILLINOIS ST ATE BOARD OF EDUCATION 

DIRECTIONS FOR MARKING: 
I Use #2 pencil only-erase cleanly. 
I Mark only one answer per item. 
I Fill circle completely. 

improper proper, 



STUDE.1MT FORM 



1 . I am: 
O Male 

O Female * 

2. The vocational program in which I 
am enrolled is: 
O Agriculture 
O Business 
O Health ' 
O Homfr Economics 
O Industrial. ^ 

3. t am in: 
O- grade (Freshman) 
O ^Oth grade (Sophomore) 
O ^ ^th gracle (Junior) 
Ot2th grade (Senior) 

. 3th grade (College) 
O ^^th grade (College) * - 

4. Who influenced Vou the mos^O^en- 
ron m your vocational coursd? ^ 
OA eacher or Administrator 

Qf Guidance "Counselor » 
Ox^FriendCs) ^ y 

O Parent _ 

5. I am'A member of a studenrorganiz*- 
tion br dub conducted as a par^ of 
my vocationat course. 

O Yes 
O No 

6. Nave you ever taken a written test in 

school to help you explore your job 

Ir^terests? 

■ OlYos 
IP ^.^ 
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7. Last year I met with a teacher or 
counselor concerning my career: 
O More than 5 limes 
O 4 - 5 limes 
Q 2 - 3 limes 
O Once 
O Never 

8. How would you rate the information. | 14 
you have received from teachers ^r 
counselors regarding your futu^ J 
job? >^ 
O High 
O Average 
O Low 
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9. Is there a series of vocational cours- 
es that you afe encouraged to take'to 
prepare for a job? 
Q Yes 

ONo _ . h i'6, 
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10. Does your school have a placement 
' ^ ^rvic8 tfiat helps you find a* job 
r after graduation? 1^ 

^11. Oo^i^Qijl feel your teachers have J 
%vclgar1y^explaltie^^hat you are sup- 
posed totlearri in^'^oJr Vocational 
courses? ' ^ ^ ;* 

o No ^ . ^ ^ ; 



12. Are your vocational course pro- | ^18« 
viding skills you need to get a jo^? 
O Yes ' 
O No/ 



L Who would you talk to if you need- 
ed to^ind a job? . 
O Teacher 

O Guidanpe Counselor 
Q Fr.e>^) 
O Parent 
O Other 

How would you rate your voc^ 
tional teacher's knowledge of the 
job in w'hich you are interested? 
O High 

O Average , 
O Low 

. How would \ou rate your coun- 
selor's knowledge^ of the -job in 
which you are interested? 

O High 

Q Average 

O Low 

Do your vocational teachers use 
field trips or guest speakers to give 
you inore information about jobs? 
a. Field Trips ^ 



O Yes 
O No , 
b. Guest sp.eakers 
O Ves 

O No ' • , 

Have you ever had the chance to 
.* ^ 

write down for your vocational tea- 
chers how you feel about your vo- 
cational courses? ^ 

O Y e^ 

Ono' 



Can you take a vocationat'qourse 
that is usually taken by. students of 
the opposite sex? 

b Yes ^ " 

O No 



1 9. Would you take a vocational course 
that is usually taken by students of 
the opposit^sex? 

. O Yes ^ 

Ono ■ ^ 

20. When you graduate, do you expect 
to: (Mark only one) 

O got a job doing what you are learning? 

O get a job similar to the one you are 
learning? 

Q continue studying about the saime job at 
another school? 

O mak^ little use oj what you are learning?' 

• ' ) 

21. Would you recommend the vocih 
tlonat course you are taking to a 
friend? 

. ©- Yes 
O No 

22. Would you recommend the voca- 
tional course you are taking to a 
member or the opposite^ sex? 

O Yes , 
O No 

DO NOT MARK BELOW THIS LINE 



SCHOOL CODE 



Pp NOT MARK HERE 
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Legal Name of Agency 



Legal Number of Agency 



ILLINOIS STATE BOARD OF EDUCATION 
Department of Adult. Vocational and Technical Education 
Program Approval and Evaluation Section 
^ 100 North First Street 

Springfield. Illinois 62777 

SCHOOL AND COMMUNITY DATA 



1 □ YES □ NO 



Does your agency conduct an annual, formal, locally directed evaluation of the vocation^ education program^ 



2 □ VES □ 



NO 



Does\our agency's locally directed evaluation system include utilization of anyifif the State Board sponsored Locally 
Directed Evaluation nnaterials? *' ^ 

If yes. check applicable activities listed below 



□ Developing an Evaluation System 
D Student Follow*Up Survey 

D Employer Follow'Up Survey 

D Student Evaluation of Instruction 

□ Assessment ofSt udent Services 

□ Assessing Student Career Instruction 
D Assessment of Instructional Materials 
D Evaluation of Facilities 



D Int^nal/External Team Review 

D Evaluating the Career Information Program 

U Personnel Evaluation and Developnr^ent 

D Identification of Occupational Competencies 

D Assessnnent of Services for the Disadvantaged and Handicapped 

D Assessment of Student Attainment of Objectives « 

D Analysis of Community Resources 

□ Cost/Outcome Analysis , 



■A- 



3. □ YES □ NO 



Is an annual evaluation report prepared summarizing the results of data gathered in vocational programs? 



Who prepares the annual evaluation report?^ 
If yes. who receives copies of the report? 



□ Board of Control * ' 

LI Advisory Councils/Committees 

□ Occupational Staff 



□ Student Services Personnel 

□ Agency Administration 



4. □ 



NO 



Does your agency utilize job market demand information rn planning vocational programs? 
If yes. what specific materials are utilized? 



S. What was last school y6ar's dropout rate for all high school students? (Not aopffCTble for post-secondary) 
Dropout rate is equal to the percent of all school leavers other than trapsferiT 



6 □ YES □ NO 



Does your agency have a formal system developed to assejs^ivdent achievement of occupational competencies? 

If yes, What kinds of instruments/measures are used? -^J 

Sample instruments might be provided to the evaluatio9rteam leaded. ^ 

\ 



7 □ Y^s ^ O NO Does your agency provide adult vocational education programs designed to serve members of the community? 

»f yes. identify any spectel programs provided by the agency to serve displaced homemakers or workers re-entering the labor 
market 



A_ 

8 indicate the number of students involved in each vocationat student orgamzatton avaii^bie to students m your agency 



NAME OF ORGANIZATION 


MEMBERSHIP 


^ SPONSOR'S NAME 


FFA (Future Farmers of America) 






FHA. ^ 

HERO (Future Homemakers of America - Home Ec Related Occup.) 






FBLA' Beta Lambda) . 
(Future Business Leaders of America 






- 

AiASA (American Industrial Arts Student Association)^ 






VICA ^ * (Vocational Industrial Clubs of America) 






DECA» (Distributive Education Clubs of America) ^ 






IDEA (Illinois Office Education Association) 






HOSA (Health Occupations Student Association) 






Other 


* ; 











« O Yes D No 1. Enrolled students (summer and/or part timelJlacement)? 
* □ Yes □ No 2. Dropouts/Schj^ Leavers? 
□ Yes □ No 3. Graduates? 

b) Who IS responsible for placement and in which department is the placement service located? 



10. □ Yps □ No Are the agency's >(pcational programs and services free of sex bias/sex role stereotyping and culturaldifferences? 
^ 

11. □ Yes □ No . ,^vl^'ha7Sl^havITM0*^^^^^^ ^'''1' ^^iM training (typically grades 1M4) have documentation on file to ^ 

Show that they haye 2.000 hours of non^ucational work experience applicable to their in$tructiQ(ial assignment? 

i - 

f 2. O Yes □ No Have any formal processes been instituted by your agency to aid in identifying the vocationahnservfce needs of staff? 

t yes. what specrtic types of staff development/in-servico activities are conducted, ejg.. State Board in-service funding 
IRequestjor Application), State Board consultant visitations, etc.? 




r 



What percent of time is allocated to the person in charge of fulfilling vocational education administrative duties, efl., work with advisory councils/ 
committees, oreperatlon of. the Local Plan and vocational claim forms, planning and evaluation, etc? oavisory councils/ 



14. □ Yas □ No ^^'^.{|^« ""^^ Year "Plfln Developer" have ^vocational education baVground. e.g., vocational instructor. If yes. ptease 



• 4 



15" What percent of the total agency budget is allocated f or vocational education program purposes? 



ERIC 



16. How would you rate the extent to which fmancial resources are allocated by your agency to support quality vocatittna! education program v"* 

> 

□ High Quality programs are not hindered by the lack of financial resources allocated for vocational programs 

D Moderate It isdiff icult to rpaintaip quality prbgrams given the financial resources allocated for vocational program; 

D Low. Quality programs have declined due to the lack of financial resources allocated for vocational programs 



7. If additional federai/state financial resources were made available to support yOur agerkoy s vocational education program, how would these resOijrces be 
best utilized? Rank order from 1 to 5, 1 being highest. / 

c 

' Equipment * *^ 

Expanded Program Offerings 

Facilities 

Materials and Supplies ^ " ~' — — - . 

Salaries , . ' , 

" ' Other (please specify) 



18. D VES D n£ Do citizens from your community serve on vocational education advisory councils/committees? 
If yes. 1$ there'. , D Ves D NO onp advisory council fqr all of the agency's vocational programs? 

^ D YES D NO a separate committee for each vocational program? 

uy the vocational programs having advisory committees, the number of meetings held per year by each committee and the names of agerfby staff who 
work with each committee. > ^ 



19. indicate w<th a check who appoints the vocational advisory councils/committees m your agency and to whomJhese councils/committees report. 





- Vocational Advisory 
Councils/Committees 
APPOINTED BY: 


Vocational Advisory 
Councils/Committees 
REPORT TO: 


Board of Control ^ 






Chief Agencv Administrafdr 






Vocational Planner/Director/Dean 






. Departrmnt Chairpersons" ^ 






Vocational Instructors ^ 






Other (specifyl 

























20 Check the advice and assistance types of functions which your agency's vocational advisory councils/committees are asked to perform 

□ Advising on development of vocational prograrri policy 

D Determining vocational program goals and requirements ^ ** 

□ Developing vocational objectives for instructional pro-ams * 
P Evaluating the vocatioijal program with respect to employment needs 

□ Advising school administrators on quahfications needed by specialized vocational instructors 

□ Assisting long term planning for vocational programs including: curriculum, equiprpeni and facilities 
^ □ Proniotmg vocational programs and services to students and ihe community 

□ Other (please specify) 

* o 

21 □ YES ^O^NO Does your agency participate in the Compreherisn/e^mployment and Training Act (CETA) program or with other state and 
locally^unded vocational training programs? 



I^f yes, specify the extent of the coordination^ 



22 What agency m your imnriediate erea is responsible for progra^ns funded under the Comprehensive Employment and Training Act (CETA)? 

U?^ I^.'^/^ctV^* workiiy relationship that exists between the vocational training provided by your agency and the ,ob training provided through 
your locai ocia rupresentatiya? » » 

a. □ HItih. Training programs and services are regularly coordinated. 

b. D Moderate. Some training programs and services ere coordinated. w 

c. Q Unitted. Most training programs and services are not coordinated. 

d. □ No coordlnatton of programs and wrylces is underway at this time. 

' ^ ^ ^ V 

24. What perbentage of your student population (uhduplicated count) IS enrolled in vocational education pro^^ * * ' 



25. Total number of studants enrolled in* cooperative education programs 



26. CURREfSIJ UNDUPUCATEO ENROLLMENT Provide the current'^unduplicated enrollment for each .of the vo£ational areas by grade level for your 
agency. (Student I should not be counted In more than one class). . 





ACmCL 


ILTURAL 


BUSINESS. 
MARKETING AnD 
MANAGEMENT 


HEALTH ^ 


^ h6me' 

ECONOMICS 


INDUSTRIAL • 


TOTAL 




tn- 
Oittrict 
Enwtl* 
ments 


. Joint 
Agre*- 
nnantt 
(Include 

. AVC) 


tn- 
Olttrlct 
Enroll- 
ments 


Joint 
Agree* 
mentt 

(Include 

AvcT 


ln< 
District 
Enroll- 
mentt 


Joint 
'Agree- 

^ ments 
Include 
AVC) 


In* 
District 
Enroll* 
ments 


Joint 
Agree 

^ mentt . 
(Include 
AVC) 


In- 
Dlttrlct 
Enroll* 
mentt 


Joint 
Agree 

mentt 
(Include 

AVC) 


In- 
Dtttrict 
Enroll- 
mentt 


Joint 
Agree- 
mentt 
(Include 
AVC) 


K.8 












t 














9th Grrfde . 
























• 


10th Grade 


























ttth Grade 










tT 
















12th Grade 


























Post-Secondary • 












* 














Continuing (AdUlt) 






.^'H 

fi 








m 1 












TOTAL ' . 






























✓ 






* • 
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ID # 
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On-site Evaluation Instrument 
Team Leader Form 



Agency Name: 
Sate: 



Pleaae circle one code number for each queetion imleee othenoiee epecified. 

1. Rate the extent' to which personnel related to the. vocational program 

(administrators, guidance Instructors, etc.) are Involved In preparing the 
^One and Five Year Plan for Vocational Education. 

Extensive. Appropriate personnel are Involved,. . . 1 

Moderate. Some personnel are Involved 2 

Limited. The |Plan Is prepared with minimal Input 
from staff , 3 

2. How would you rate the vocational education programs of the agency? 

High. A very extensive and comprehensive program 
Is being offered 1 

* • 

* Average. A program exists but Is not serving all 

who (^ould benefltf from such^^an experience .... 2 

\ 

Low. A minimal program exists whtfih should be further 
developed and refined .... . T\ 3 

3. Is occupational information provided In elementary feeder schools? 

Yes. Most feeder schools are providing occupational 
Information to students i 

Some. Only some of the feeder schools are providing 
occupational Information to students 'z 

No. None of the feeder schools are providing 

occupational info^imatlon to stud'ents 3 
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4 



10 



1 1 



4* Rate the extent to which agency personnel are Involved with articulation 
efforts within the Institution and with other Institutions (K*Adult)* 

Extensive* Agency personnel actively encourage and participate 
In vocational program articulation activities . 1 

^ Moderate* Articulation efforts appear to be sporadic and 

without coordination *^ 2 

Limited. Minimal articulation actlvltles^re underway at 

this time r 3 

5a* Does the agency claim disadvantaged, handicapped or limited English profl* 
clency atudents? 

* 

Yes * * ; 1 

No (Skip to Q.6) . . . 2 

b* Does the agency Identify disadvantaged, handicapped or limited English pro- 
flclency^students as stated In the Local Plan? 

Yes * 

^ No 2 

c* Does the agency serve disadvantaged, handicapped or limited English profi- 
ciency students as stated In the Local Plan? 

Yes 1 

No 2' 

, In some Instances . . 3 ' 

6. Does the agency place handicapped vocational education students In the least 
restrictive educational environment? 

Yes 1 12 

No 2' 

In some Instances . . 3 ^ 

7. How would you rate the effectiveness of existing vocational student organi- 
zations within the agency. 

High. Most organizations are effective .... 1 13 

Average. A few of the organizations are 

effective but most are minimally effective . ^ ^ * 

^ " Low. 0rganl;satlon8 are very limited In member- 
ship and attendence ..... 3 
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8a. To what extent is the vocational advisory council/commit tee(8) in the agency 



> 



composed of representatives of buslneqs, Industry and lab^r? 

High. The counell/commlttee(s) represents 

business. Industry and labor 1 

Modetate* One of the areas (business. Industry 
and labor) Is not represented on the council/ 
conimlttee(8) , 2 

Limited. More than one of the areas ^business, 
Industry, labor) Is not represented on the 
coancll/committee(s) 3 

*b. Do the advisory councll/commlttee(8) have representative numbers of women 
and men consistent with the area and programs served by the agency? 

Yes * 1 15 

No 2 

In some Instances • • 3 

c. Do the advisory councll/commlttee(s) have representative numbers of racial 
and ethnic minority groups consistent with the area served by the agency? 

^ ^ Yes ./..lie 

No ....... .j . r .> 2 

In some Instances . . 3 

9. Rate the extent to which the agency utilizes the advisory commlttee(s) . 

^ High. Agency personnel utilize the advisory 

commit tee(s) extensively 1* I7 

1 

Average. Agency personnel utilize the advisory 
^ commlttee(s) on a limited basis 2 

Low. Agency personnel do not utilize the 

advisory committee(s) 3 

ft ' 

10. Is the vocational program of the agency coordinated with programs sponsored 
under the CoiJ5>rehen8ive Employment and Training Act (CETA) and other public 
and private training programs offered in the area? 

Yes. Agency personnel cxoordinate programs? with 
^ outside agencies . - 1 i b 

Limited; Annual contact is made 2 

No. Programs are not coordinatetd 3 
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11. Does the agency have a locally directed evaluation* system? 

^ Yes. A system exists which/enables Evaluation 

of the total vocational program 1 -19 

Under development. The development of a locally ^ 
directed evaluation syst'em Is underway , • • • 2 * ^ 

No.^ An organ^ed system does not exist other / 
than testpig^done by Instructors or evaluation 
of staff/members conducted by the admlnl- 
stratori» etc 3 

12ar How would you rate the quality of the cooperative education program offering 

In the agency? ^ ^ 

High. . Training agreements and plans are In evidence and 
adequate time Is allotted for supervision of students ... 1 20 

Average. A program exists but Is only minimally serving the 
students enrolled • ' 2 

Low. Considerable revision Is necessary to Improve the present, 
program offerings '. . . . 3 

A cooperative education program Is not offered by the agency 
(Skip to Q.14) . 4 

b. If training agreements exist » dc^^ey contain a statemenlj^ that assures non- * 
dlacrlmlnatlon on the basis of s6x, handicap, race and national origin? 

^ Yes ......... 1 21 



No 2 

^ In some Instances . . 3 

13. If a cooperative education program Is offered, are students placed^^l^^ 
training sites regardless of race, color, national origin, sex or handicap? 

Yes * 1 22 

No 2 

14. To what extent are vocational programs offered by the agency focusltig ^on up- 
V to**date occupational competencies? 

Extensive. Programs are regularly reviewed and revised based 
on changes In Job* training needs . . 1 « 2 3 

0 Moderate. Some programs In the agency are, regularly reviewed 
« and revised to better meet job training needs 2 • 

^sl"' ' Limited. Agency personnel do npt review and revise programs 
* 4r . on a regular basis- • . • . . 3 



15 • Indicate your perception pf the decree of Involvement the agency has had 
with the State Board-sponsored Career Guidance' Center In the region* 

High. Regular contact Is evident • • • 1 

Moderate. Staff has attended one or more Career Guidance 

Center meetings during the ^ast year 2 

Limited. * Staff receives Center newsletters regularly ... 3 

No Involvemenjt: evident 4 

16. To the best of your knowledge » Indicate the Career Guidance Center services 
that the^ agency has utilized. (Circle all that^apply) 

Newslii^terft^.-«...,^^jy^ . . ^ . . 1 

In-service training ..... 2 

^ Resource materials 3 

Placement assistance or 
f ' Information 4 

^ Consultlve services .... 

Special assistance ...... 6 

^ . ^ None of the above 7 

» 

17. Do the vocational programs offered thifough the One and Five Year Plan for 
Vocational Education afford equal access for all students to all^ programs 
proposed by the agenc^r thus assuring that no student will be denied access 
to such programs on the basis of race» color^ national origin^ sex or ^ 
handicap? 

^ Yes 1 

No 2 

y 

Ida. Is there a policy statement prepared ^by the agency and approved by the 

Board of Control Insuring equal educational opportunity regardless of race» 
color > national origin^ sex or handicap? 

^ Yes . 1 



No (Skip to Q.'19). . 2 



b* Has the p'ollcy been dJLssemlnated to staff with implications for performing 
Job respon8ibilit;ies? ^ 

* Yes .1 

, .1- , No 2 

c* Dqes evidence exist of staff compliance with the approved policy? 

((^ J Yes 1 

No' 2 V 



19* Does the afgency provide public notification of its non*discriminatory 
practj£^«T 



Yes 1 

No 2 



I J \ 
20* Has any locally directed (self) evalu^J^on been conducted l>y the agency to 



assess the existence and the possible extent of sex bias and sex rple 
stereotyping in vocational programs? 



Yes 1 

No 2 



21* Does the K*8 occupational information program offered by the agency address 
the Issue of sex role stereotyping (i*e*» sex fair language and materials, 
bulletin boards etc.)? ^ V 

Yes • . 1 

: No. ^ 2 

. The Agency does not offer a K-8 

Occupational Information Program • • 3 

i 

. ■« . ' 

22. Is sex fair language used by the agency In all publications related to voca 
tional edtjcation (i.e., student handbook, Local Plan, course descriptions, 
etc.)? 

Yes 1 

t No 2 
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23. Rate the extent to which guidance and vocational orientation activities In 
the agency are ^provided to students to encourage them to consider all 
programs of study, regardless; of their sex. ^ 

Hlglv; ^ The Importance of non-blased career planning Is ^ 
s dressed for both male and female students 1 

Medium. Materials and resources utilized are generally sex 
fair but little time Is spent In counteracting undue 
social pressures to conform^ ^ # 2 

Limited. No effort Is made 3 

24 » Is participation In agency ancillary services such as guidance" and 

counseling, students organization, placement services, etc., open to all 
students regardless of race, color, national origin,, sex or handicap? 

. • < Yes 1 

» , No 2 

25. Have staff of the agency (Instructors, administrators, guidance personnel, 
etc.) received in-'service trainings -related to the elimination of discrimi*- 
natory practices regarding race, color, national origin, sex, and handicap? 

Yes * I 

No 2 

26. Which of the following groups have received in*~service as an aid in 
exploring issues related to sex equity . (Circle all that apply) 

Instructors I 

v., 

Aclministrators 2 . 

Student services personnel . . . .* . ' 3 

Cooperative education coordinatfors* . 4. 

Advisory council members I 

I 

, .J Board of control members 2 

• ^ ' 

Parent groups 3 

Community groups . 4 

. V ' 
None of the 'above 1 
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Coder^ 



Keypunch 
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»»2 



t»3 

t»5 
t»6 
t»7 
k 8 
t»'9 
50 

52-5»* 
'55-57 



5 B'^ 9 /BK 
80 /l • 



EMPLOYER FOLLOW»UP SURVEY 



. Sufv^'y NumbtT 



INSTRUCTIONS/ Please answer the foUowmg questions and return (L^form m the enclosed pre addressed stamped envelope 



A. VOpATIOfJAL TRAINING EVALUATION 



Please rate the vocational training received by the mdividual in each of the following areas: {>/) 

very Good Gt)od Neutral Poor very Poor 

U) - (2) , (3) (4) \ (5) 

' t Technical Knowledge. | | [ - [ [ [ [ | | | 

2 Work Attitude • QJ* |~j 

a Work Quality [Zl • (Zl [^1 ED [~| 

OVERALL RATING ^ ^ ^ ' 



Check the overall rating of the vocational training- received by th.s individual as it relates to the requirements of his or her job: 

<• Very Good Good Neutral Poor very Poor 

JIL iiL (5) 



RELATIVE PREPARATION 



'^j'''^^* person's vocational training; please rate his or her preparation in relation to other employees m the work group 
who did not receive such training: {^) . a 



I I <n Individual is better prepared 

I I <2) IndividuahsName as othV employees 

' I ] (3) l-ndividual is less prepared ^ 

^ I ] (4) No* basis for compariso/i 



/ 



COMME-NTS: 



^ . THANK YOU FOR HELPING US. PLEASE RETITRN THIS FORM IN THE ENCLOSEO ENVELOPE. 

IFR 



STUDENT FOLLOW-UP SURVEY 



Survey Numt^^r 



INSTRUCTIONS Please answer the following questions and return this form tn the enclosed pre addressed starpped envelope. 



A. Scho6l Status (check only one box) 
I I 1 I am not in school 



□ 



*2 I am in school taking classes to become a. 



B. Work Status (check only one box)' # * 

I | > 1 I am in the fuM-time military. 

\ j 2 I have a job and. I am not in (he full-time'nnilitary. 

I I 3 I am unemployed a^d looking for a job. 

I I 4 I am unemployed and not looking for^a job. 



NOTE If you have a job or you are in the full time military. 
, follow-up form in the enclosed envelope. 



please answer the foIlOAtng questions if you are unemployed, stop here and return thts 



C. Please provide the following information. 



JOB TITLE. NAME OF JOB. OR MILITARY TITLE 


NAME OF BOSS. SUPERVISOR OR IMMEDIATE 
COMMANDING OFFICER 


DESCRfBE THE WORK THAT YOUt)0 















f 


NAME OP COMPANY WHERE YOU WORK OR MILITARY INSTALLATION WHERE YOU ARE STATIONED 


STREET ADDRESS \ . ' 


CITY 


STATE 




ZIP CODE 



D. h the work that you do related to the vocational classes you had in school? 
^ 1 I 1 Yes. I am using my vocational training. • 

I [ 2 No. I am not using my vocational training. 



E. How rnany hours do you work each week? (Do not include overtime) 



hours pef week 



F How much do you make per hour? (Do oatinclude overtime pay) 



per hour 
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THANK YOU F.OR/lELPING US. PLEASE RETURN THIS FORM IN THE ENCLOSED ENVELOPE. 



VITA . . 

Juan C. Gonzalez was born on March' 8, 1952 in Amarillo, Texas 
where he completed -his elementary and secondary- education. He re- 
ceived his Bachelor of Arts degree from.-Texas Tech University, 
Lubbock, Texas, in 1974, where he majored in Latin American 

Studies! He earned his Masters of Arts degree in Bilingual-Bi- 

\ 

cultural Studies from the University of Texas at San- Antonio at 
San Antonip, Texas in 1976, Currently he is a Title VII Bilingual 
Teacher Tral^r Fellow^t the University of Illinois, working toward 
his Ph.D. in. Educational psychology. 

He has had extensive experience in the evaluation ofbilfngual 
education programs and in testing of minority children. In addi- 
tion, he has worked as a career counselor fo^ minority studentsOv 
both .in Texas and at. the University of Illinois. 
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